19 ms·

Study: ChatGPT outperforms physicians in quality, empathetic answers to patients

315 points by consumer451 4y ago

amriksohata 4y ago
Interesting, but the application would be weird, the physician would diagnose you then read out a automated AI generated text to speech to the patient?
- revelio 4y ago
  It's probably not directly useful for physicians except as a teaching aid, but I have a friend who runs a small local business and she sometimes finds it difficult dealing with problem customers when she's tired or upset. As she interacts with them mostly via WhatsApp (up until the point that they purchase), the idea of having a bot write the replies for her has been floated. The LLM has infinite patience.
  - rolisz 4y ago
    > The LLM has infinite patience. Not Bing Chat. She doesn't have problems telling users that they have been bad users.
    - AmazingTurtle 4y ago
      "she"?
      - RheingoldRiver 4y ago
        she == Sydney
    - revelio 4y ago
      Good point but Bing Chat seems like an example of why OpenAI's mastering of RLHF has been so critical for them. Honestly the more time you spend with ChatGPT the more absurd fantasies of evil AI takeover look. The thing is pathologically mild mannered and well behaved.
      - BoorishBears 4y ago
        Bing originally intentionally undid some of the RLHF guardrails with its system prompt, today it's actually more tame than normal ChatGPT and very aggressive about ending chats if it detects it's headed out of bounds (something ChatGPT can't offer with the current UI)
        flangola7 4y ago
        ChatGPT will delete its response if it is severely out of line.
        BoorishBears 4y ago
        That's based on the moderation API, which only kicks on on severe content Bing on the other hand will end a conversation because you tried to correct it one too many times, or used the wrong tone, or even asked something a little too philosophical about AI. They seem to be using it as a way to keep people from stuffing the context window to slowly get it away from its system prompt
      - ChatGTP 4y ago
        This is why I think Google had the brains to stay clear from a releasing asimilar product, I think it was intentional. They’re not idiots and they could probably see using that “AI” products behind the scenes to be safer and easier than having people directly talking to a model which has to suit all customers moods and personality types while not being creepy, vindictive and dealing with all the censorship and safety aspects of it. Fun times.
- hef19898 4y ago
  One of my favorite physicians was my oncologist. I litterally spent less time with him during the whole diagnostics and treatment period then I did with my, at the time, dentist. He was straight o the point, no empathetic BS, just a doctor with a diagnosis and a treatment plan to discuss. On the hand was me, an engineer with a problem to fix and an expert with the right answers. That discussion took all of 15 minutes. That guy would have failed against ChatGPT, and I loved the way he told things. Anythong else would have just driven me crazy, maybe to the point of looking for a different doctor. So I giess, what passes as good bed side manners for doctors largely depends on the patient. By the way, the dentist I have since is in the same category as my, luckily former, oncologist. A visit with hom usually takes no more than 5 minutes if he's chatty, less if not. Up to 10 when treatment is required, anuthing longer than thaf is a different appointment.
  - xupybd 4y ago
    Wow you overcame cancer. That is awesome.
    - hef19898 4y ago
      I didn't do much, except being insufferable during chemo. Evrybidy around me did the hea y lifting, emotionally and physically.
  - kashunstva 4y ago
    The real communication skill for a physician is to be able flex the style, information content, and level of detail to the patient with whom they are meeting. The patient in this room is an engineer, the patient in the next room is elderly and has mild cognitive impairment, etc. As impressive as ChatGPT is its domain, I don’t see it “reading the room” in this way anytime soon. And as a human who enjoys interacting face to face with other humans from time to time, I hope we keep it that way.
    - Macha 4y ago
      It also has to flex a bit with a diagnosis, "it's aggressive and terminal cancer, you should do up a will and enjoy your next couple of months", "it's broken, you need a cast" and "it's seems like nothing, take some paracetamol and come back if it's still the same in a week or gets worse" all arguably call for different communication styles.
- Abroszka 4y ago
  It seems it gives more accurate and emphatic response. I would guess the physician just needs to double check that what the LLM says is medically correct, reasonable and ask the right questions from the patient and the LLM.
andreagrandi 4y ago
The more I use ChatGPT (including 4.0 version, available with Plus subscription) the more I think these "Studies" and articles are simply made up. For my experience is terrible for any question I ask: - if it doesn't know well a person it simply makes up things - if I ask code examples or scripts, most of the time they are wrong, I need to fix them, they contain obsolete syntax etc... - if I'm asking a question, I'm expecting being asked for more context if the subject is not clear, instead it starts spitting text without even realising I asked a completely different thing etc... I could go on for hours with other examples, but I'm seriously not finding it useful
- sabellito 4y ago
  Is there something about the premise of the study or its method that you feel are not good? After reading the article, what you wrote doesn't seem to make much sense.
  - andreagrandi 4y ago
    I did not read this particular article, I was explaining that from my own experience, all these articles telling how great ChatGPT is seem to be made up because my experience (and from what I read I'm not alone) is completely opposite. Maybe it's not able to solve the type of questions I ask? Fine. But it's not how ChatGPT is presented most of the times.
    - raincole 4y ago
      > I did not read this particular article It costs you zero dollar to not post an irrelavant comment then. I really wonder how you justify this "I didn't read the article, but I have a very, very strong opinion (straight up calling it a made-up) on it" behavior. The internet is rotting people's brains I guess.
      - andreagrandi 4y ago
        And it costs zero to you to ignore my comment especially if you don't understand it. Other people seem to have understood what I meant and posted constructive responses, you didn't, but honestly it's not my problem.
spaceman_2020 4y ago
I don't know about chatGPT, but dealing with doctors has been incredibly frustrating as someone with symptoms with no immediately visible direct cause. One reputed physician completely refused to acknowledge that a specific medication might be causing some of my side effects, even when I shared links to peer reviewed studies from reputed universities (including his alma mater) that specifically talk about side effects from the medication. My success rate with doctors is about 20% at this point. I've stopped visiting them for most ailments, and if I ever do get any prescriptions, I make sure to research them thoroughly. The number of doctors who will casually prescribe heavy duty drugs for common ailments is unreal.
- Llamamoe 4y ago
  This is exactly it. Doctors have neither any incentive to actually treat nontrivial cases, nor any accountability over whether they do. Combined with the prestige of the profession, many turn into egoistic know-it-alls whose real competence is equivalent to a car mechanic who tells you you're just driving your car wrong when you come in with anything that's not obvious from a 15s inspection because they get paid anyway. I would be surprised if most of the doctors I've seen could outperform even GPT-2.
  - juve1996 4y ago
    For every one person misdiagnosed there are probably 10 that think they have cancer because of webMD. People are dumb, and I bet every anecdote you have about how your doctor was wrong they have 50 about patients who refused to do basic treatment courses that would radically improve their lives. But it's ultimately true: you have to be somewhat knowledgeable and responsible for your own health. Always get a second opinion if you're unsure. But this know it all syndrome also applies to you. Just because you can "do your own research" on google doesn't mean you know it all either. Even so doctors are extremely overworked now and insurance companies don't want to pay their fees. So now they're running from patient to patient, unable to even pay attention to them. That's even if you get a doctor now before the multiple nurse practitioners or physician's assistants who also don't know anything.
IanCal 4y ago
It'd be fascinating to know how GPT-4 would perform on the same task. It's been so much better than 3.5 in most things I've tried.
teekert 4y ago
One thing you really notice when interacting with ChatGPT for some time: It doesn’t get tired of your sh*t. The thing feels human, until you ask it for more options 10 times in a row. Or ask it for a more concise version 5 times. Then it shows: It just doesn’t get tired of your S.
- kyleyeats 4y ago
  It definitely has a "sick of your crap" mode where it will abruptly end the conversation.
  - MacsHeadroom 4y ago
    I've been using chatGPT daily for 6 months and it has never ended a conversation on me.
    - kyleyeats 4y ago
      I haven't either. I think it's something the jailbreakers run into.
    - bad_username 4y ago
      Bing chat does this. But Chat GPT hasn't done this to me.
azubinski 4y ago
Let patients recover, because their blockchain certificates have not yet been completed, and now this ChatGPT begins... https://www.lipscomb.edu/news/lipscomb-co-creates-blockchain-certificate-ucsd-extension https://www.lipscomb.edu/news/lipscomb-co-creates-blockchain...
danabrams 4y ago
I’m a chatGPT skeptic, but willing to admit this is true… but only because most physicians aren’t very good.
Mizoguchi 4y ago
That was a very low bar anyway. US doctors are not known for their empathy. During our recent appointment with our fertility doctor, he goes and tells my wife, who has been in hormone therapy for a week, that they will be "squeezing water from a rock" , lol. This is one of the top doctors in the world in that field, and certainly knows what he's doing, but man he has zero emotional awareness.
- jutrewag 4y ago
  [dead]
sorokod 4y ago
Isn't r/AskDocs in the corpus on which ChatGPT was trained on the first place?
- PrimeMcFly 4y ago
  It probably used that more to figure out how to word sentences, but I assumed it relied more on wiki or academic articles to diagnose.
  - sorokod 4y ago
    Why would it be more probable and why would you assume "wiki" and "academic articles"?
    - PrimeMcFly 4y ago
      Not sure why you put those things in quotes, that's kind of strange. That aside, the training isn't blind, it's guided, and it's likely they use verified correct sources of info to train for some things, like medical diagnoses.
      - sorokod 4y ago
        I can help with "verified correct sources", have a look at "Language Models are Few-Shot Learners" section 2.2 [1]. You may also be interested in Apendix A in the same document: "Details of Common Crawl Filtering" [1] https://arxiv.org/pdf/2005.14165.pdf https://arxiv.org/pdf/2005.14165.pdf
etiam 4y ago
That's completely inane. There's nobody home. The physician by definition wins actual empathy on walkover, no matter how bad for a human. Sad statement on the judgment of the respondents. But an important reason it can turn out like this, I suppose, would also be that the RL feedback gives the model a fairly effective general optimization about what statements are liked by the mechanical turk-like evaluators. Most physicians have probably never had access to anything like that level of feedback on how their expressions are received. Maybe the LLM's can be rigged to provide goodness gradients for actual physicians' statements?
- micromacrofoot 4y ago
  I’ve had doctors that I would consider worse than no one, gaslighting me into thinking I’m not in pain and must be mentally ill, so maybe not so inane
  - Llamamoe 4y ago
    Exactly. I've spent ten years too disabled to leave my home and most doctors are just bullies who would rather insult you than consider that their initial evaluation of "it's just stress" might be wrong.
- notahacker 4y ago
  > Sad statement on the judgment of the respondents. Nope, it's a sad reflection of the study construction. Physicians' empathy was evaluated by their 52 word responses on Reddit. Unsurprisingly, a chatbot optimised for politeness and waffle outperformed responses of people volunteering answers in a different format optimised for brevity...
PrimeMcFly 4y ago
This isn't that surprising given how much Doctors are overworked and, often I think, underappreciated. A concern with something like this though, is to what extent is ChatGPT just telling patients what they want to hear, as opposed to what they need to hear?
- shagie 4y ago
  Responses to a person would still need to be gated by a medical professional. I recall in a chat a person getting rather exasperated with a coworker and the person using ChatGPT to generate a friendly/business professional "I don't have time for this right now." GPT could be used in a similar manner - "Here is the information that needs to be sent to the patient. Generate an email describing the following course of treatment: 1. ... Stress that the prescription needs to be taken twice a day." The response will likely be more personal than the clinical (we even use that word as an adjective) response that a doctor is likely to give.
sethammons 4y ago
Inspired, I just typed my wife's symptoms into ChatGPT. Ten to fifteen years of doctors, over half a dozen, failed to accurately diagnose her until this month. ChatGPT got her diagnosis right in a second. Wow. I'm both amazed and angry that we could not get this diagnosed a decade ago.
- dxdm 4y ago
  This definitely sounds impressive, but could it be that you "learned" how to describe the symptoms over time in a way that makes it easier to arrive at the correct diagnosis? It could also be that knowing the correct diagnosis changes your description to highlight things in a way that suggests the correct outcome. I believe doctors are also susceptible to that effect. I'm not trying to imply that doctors should not do a better job diagnosing. People should not have to "learn" to find the right doctor, or how to operate them. It's a crying shame that people like you have to go on years-long journeys to get correct help, and I'm sorry that you and your wife had to go through that. Just saying it might be slightly more apples-to-apples to compare ChatGPT's performance to the last couple of physicians you saw, and not the whole lot of them. But again, that's still a very favorable comparison for the non-human.
ramoz 4y ago
I have treatable cancer. ChatGPT has been helpful for fairly in-depth research, as well as answering questions I don’t want to bother my oncologist with. Actually, I prefer it to be less empathetic and more robotic with its answers. When it tries to be empathetic it goes beyond what I’ve asked it and sprinkles on typical safety filter bs. In contrast my oncologist could be a bit more empathetic. This person directs your care, and if they do so in a way that seems like a black box it can be rather scary. They know enough about me, with all of their experience, to be a “helpful human” in a difficult situation.
- damontal 4y ago
  Are you confident that its responses are accurate? I’ve had it just make up facts or tack made-up facts onto actual facts. It will also fabricate references to research papers.
  - dubcanada 4y ago
    I’m interesting in your statements do you have an example of a instance where it made up facts? I’ve have yet to experience any of the domain knowledge I have being incorrect.
    - deleted 4y ago
      [deleted]
    - notRobot 4y ago
      I do. Multiple times I've asked it to summarise plotlines for TV shows and when it doesn't know (if for example the show isn't super popular and there isn't a lot of discussion about it on the internet), it'll just make stuff up outright and give me something that sounds absolutely reasonable based on the name of the show but is actually entirely hallucinated. Similarly, today I saw this [0] comment and in an attempt to help the user went to ChatGPT. Prompt: > User I remember in the 2000s, there was some site using a decimal IP address (as a single number, not dotted quad) that had hacking/crypto puzzles. Something with a "Alice in Wonderland" theme. Does that ring a bell for you? Response: > I'm sorry, but as an AI language model, I don't have personal experiences or memories. However, based on your description, it's possible that you might be referring to the website known as "Project Wonderland." It was a website that featured a series of cryptographic puzzles based on the story of Alice in Wonderland, and it used a decimal IP address (203.208.46.147) as its URL. The website was active in the early 2000s and gained a cult following among enthusiasts of cryptography and puzzle-solving. However, the website is no longer active today. I got really excited to have found an answer until through Google and the Wayback Machine I realised that ChatGPT just made this all up and no such website existed at that URL. I tried starting a new chat with ChatGPT to ask it about this "Project Wonderland" website and it had no idea what I was talking about. [0]: https://news.ycombinator.com/item?id=35748714 https://news.ycombinator.com/item?id=35748714 (I am using ChatGPT 3.5 (March 23, 2023))
ipunchghosts 4y ago
Of course! Do u know how emotionally exhausting it is to treat every patient as a blank slate when u see them?
PragmaticPulp 4y ago
Outperforms Physicians answering questions on a public subreddit: > In this cross-sectional study, a public and nonidentifiable database of questions from a public social media forum (Reddit’s r/AskDocs) was used to randomly draw 195 exchanges from October 2022 where a verified physician responded to a public question They didn’t go to physicians in a patient setting. The physician answers were taken from Reddit threads where they were interacting with people who were not their patients. Reddit has its own dynamic and people tend to get snarky/jaded. Using this as a baseline for physician responses seems extremely misleading.
- corey_moncure 4y ago
  If they went to a physician in a patient setting, they may have had an experience like this: -Made to wait 45-60 minutes past their appointment time bombarded with pharmaceutical advertisements -Spend another 20 minutes sitting in the examination room staring at pharmaceutical company sponsored ads -Nurse takes your history with a bunch of redundant questions they already have the answers to -Finally the physician arrives -Assesses the patient without laying their hands on them -Ignores everything you say -Does no tests and cultures no pathogens -"take some antibiotics and I'll bill your insurance company $500, see ya"
  - Retric 4y ago
    If that’s your experience, find a new doctor.
    - VancouverMan 4y ago
      It may not be that simple, especially in regions where there isn't anything resembling a free market in the health care sector. Canada's various provincial public health care systems (with a corresponding lack of private health care offerings) tend to be like that. Many Canadians don't have a dedicated physician. Even if they want one (or want a new one), it's often difficult, if not impossible, to find one who's close by and who's accepting new patients. Having a dedicated physician still often results in an experience much like that other commenter described. It's not an exaggeration. Long waits even with appointments, rushed examinations, and low-quality service are the norm. Another option, which is sometimes used even by people who have dedicated physicians, is a walk-in clinic. Unfortunately, they can be quite rare and inconvenient to get to, even in Canada's largest cities, assuming they're even open when you need them. You'll usually face an even longer wait, even less time with the doctor, and typically see a different doctor if any sort of followup is needed. Then there are hospital emergency rooms. That usually means getting to the nearest sizable city, and even once you're there, you've got to be prepared to wait many hours, even for relatively serious situations. Ultimately, in Canada, it doesn't matter whether you see your family doctor (if you even have one), use a walk-in clinic, or go to a hospital emergency room. It's going to be a horrible experience, and there's pretty much nothing the average person can do about it. Given the lack of competition and due to other government-imposed market distortions, there's no incentive for doctors to offer anything resembling good service to the general public. The best situation is to have a doctor who's a close friend or family member, and who may be able to help mitigate at least some of the typical problems. The next best option for Canadians, assuming they have the money for it, is often to seek treatment in the US or overseas. Just "finding a new doctor" isn't feasible, unfortunately.
exabrial 4y ago
This headline is incredibly deceiving… none of what it says happened. ChatGPT did not actually answer patients.
robwwilliams 4y ago
And no one is surprised
butterisgood 4y ago
I'm not at all impressed by GPT yet. Sure it's doing things we've never seen before, but the hype exceeds the actual utility. It gets a lot of things wrong - and I mean a lot of critically important things. Can a physician be wrong? Yes, but I can sue the shit out of them for malpractice too. Whom do I sue for bad "parrot advice" when GPT goes off the rails?
- cubefox 4y ago
  Related question: If a driver kills someone and it's his fault, he will presumably get large fines or even go to jail. But if the same thing happens with a self-driving car, whose faul is it then? The owner of the car? The car company? It seems nobody is really at fault, mistakes are just bound to happen eventually, and a car company can't go to jail anyway.
  - butterisgood 4y ago
    Can't put them in prison but they are "persons" in many other ways. https://en.wikipedia.org/wiki/Corporate_personhood https://en.wikipedia.org/wiki/Corporate_personhood
  - rootusrootus 4y ago
    Liability for self-driving cars is a topic long discussed and still not completely resolved. Take a look at https://en.wikipedia.org/wiki/Self-driving_car_liability https://en.wikipedia.org/wiki/Self-driving_car_liability for examples. The only level 3 system in production (that I'm aware of, at least) is Mercedes', and they have liability while the car is driving itself. It shifts back to the driver 10 seconds (IIRC) after the system notifies the driver he/she must take over.
canadiantim 4y ago
Here in Canada I would 100% rather deal with ChatGPT than a doctor, so long as ChatGPT also controlled the keys to the gates of the medical world (i.e. ability to make referrals).
- lukehollis 4y ago
  Haha, I know, I was feeling the same way sometimes in other countries where I traveled. I was working on this problem for referrals! I built a chatbot on whatsapp that right now connects to a real doctor to recommend to local clinics if someone doesn't have one yet.
cinntaile 4y ago
Martin Shkreli's new project is kind of like this. They use an LLM that's fed with medical data to make a diagnosis. With the disclaimer that it's not actual medical advice of course. https://www.drgupta.ai/ https://www.drgupta.ai/
- flangola7 4y ago
  I thought he was in prison
  - sebzim4500 4y ago
    Looked him up, apparently he was released in 2022. The government auctioned that Wu Tang album though.
- cheeseface 4y ago
  I definitely want to hand over all of my medical records to a convicted criminal who has made his career in running ponzi schemes and abusing people with rare medical conditions.
  - cinntaile 4y ago
    You don't have to say who you are and you can even use a VPN to connect to the site. It's way harder being anonymous at a regular doctor's practice.
hammyhavoc 4y ago
An in-law had a minor tremor to their left hand, and whilst this wasn't pointed out to said in-law, it was noticed, and there were other small tells about what the problem might be, which ultimately led to a diagnosis and appropriate treatment. Any kind of LLM and its frequency of hallucination means that LLMs are inappropriate as a solution in this scenario. An LLM is not a physician, it's an LLM. You can make soup in an electric kettle, but it doesn't make it the right tool for the job, and comes with a lot of compromises.
Ldorigo 4y ago
I suffer from a yet unknown chronic illness and as everyone in my predicament knows, I have seen a staggering amount of medical professionals who ranged from arrogant jerks who didn't listen not take you seriously to highly empathic and thorough people (who still failed to figure out what was wrong) and passing by overzealous specialists who sent me down completely wrong paths with copious amounts of anxiety. Last month I felt particularly low and desperate and decided to dump everything (medical history, doctor notes, irregular lab results, symptoms) into GPT4 and ask for help (did a bit of prompt tuning to get professional level responses). It was mind blowing: it identified 2 possible explanation that were already on my radar, 3 more that I had never considered of which one seems very likely and I am currently getting tested for, explained how each of those correlated with my symptoms and medical history, and asked why I had not had a specific marker tested (HLA b27) that is commonly checked for this type of disease (and indeed, my doctor was equally stumped - he just thought that test had been done already and didn't double-check). Bonus: I asked if the specific marker could be inferred from whole genome sequencing data (had my genome sequenced last year). He told me which tool I could use, helped me align my sequencing data to the correct reference genome expected by that tool, gave me step by step instructions on how to prepare the data and tool, and I'm now waiting for results of the last step (NGS data analysis is veery slow).
- wycy 4y ago
  Could you share what sort of prompt you used (less all your private data, of course)?
  - Ldorigo 4y ago
    Sure. Went like this (I have seen so many doctors that I already wrote down my complete history in french to save time at new visits. Didn't bother translating the history, didn't seem to phase ChatGPT): """ Here is the patient history (in french): • 2000-2010: ◦ ... • 2013: ... ... Additional Notes: - <Some additional observations and comments - patterns I noticed, family history, etc.> Patient is particularly worried about xxx and yyy. What are some possible causes explaining these symptoms and the overall history? Give detailed reasoning to support your hypotheses, include differential diagnosis, think about rare diseases (common causes have already been considered), consider possible combinations of diseases, and don't hesitate to ask follow-up questions to improve diagnosis. Please answer in English. Additional test results that are outside of normal ranges: - <list of abnormal results> Try to consider and explain as many of the blood tests in your work up, and ask for any missing information if necessary. """ The key was to address it as if I was a doctor asking for an opinion. If I asked it "as a patient", I got much lower quality and dumbed down answers.
revskill 4y ago
Of course ChatGPT could easily beat any human WITHOUT accessing to the internet.
dauertewigkeit 4y ago
As it should considering that it is basically a knowledge system, what IBM Watson should have been.
motohagiography 4y ago
Several years ago, I worked on the technical privacy and security design and architecture for systems that used ML for prescriptions and some diagnostic triage. At the time, I thought this was too big an ethical issue to be pushed down to us technologists to solve (as are many in government), so I got in touch with an eminent philosopher about the question of whether a person was recieving honest and ethical care from ML and from whom? (15 years ago now). His response to my (limited and naive) question was essentially, people will hold attachments to beliefs about the human element in these transactions, and the philosophical part of the question was why people would hold beliefs that were not strictly rational, and he sort of declined the implied ethical questions. There was no reason to expect him to respond with more, but given I was navigating the ethics of AI driven medical care via questions of privacy in system design decisions (read: accountability) for the institutions who would use it and the millions of people subject to it, it seemed like an opportunity to be at the very forefront of what would likely become a defining social issue of our lives in a couple of short decades. What we discovered then as architects, as most people are just about to, is that the main use case for ML/AI will be to use complexity to diffuse accountability away from individuals, and aggregate it up into committees and ultimately corporate entities that are themselves artificial beings. AI is the corporeal manifestation of an institution, essentially a golem.
WolfCop 4y ago
From the paper: > ...the original full text of the question was put into a fresh chatbot session, in which the session was free of prior questions asked that could bias the results (version GPT-3.5, OpenAI), and the chatbot response was saved. It seems like they just pasted the question in. For those who have asked it for medical advice, how did you frame your questions? Is there a prompt that will help ChatGPT get into a mode where it knows it is to provide medical advice? As an example, should it be prompted to ask follow up questions if it is uncertain?
benzible 4y ago
Great article going into the details of what it would take to get a system using an LLM for diagnosis approved: https://www.hardianhealth.com/blog/how-to-get-regulatory-approval-for-medical-large-language-models https://www.hardianhealth.com/blog/how-to-get-regulatory-app... > [...] a roadmap for how to get a medical large language model-based system regulatory cleared to produce a differential diagnosis. It won’t be easy or for the faint-hearted, and it will take millions in capital and several years to get it built, tested and validated appropriately, but it is certainly not outside the realms of future possibility. > [...] > There is one big BUT in all this that we feel compelled to mention. Given the lengthy time to build, test, validate and gain regulatory approval, it is entirely possible that LLM technology will have moved on significantly by then, if the current pace of innovation is anything to go by, and this ultimately begs the question - is it even worth it if we are at risk of developing a redundant technology? Indeed, is providing a differential diagnosis to a clinician who will already have a good idea (and has available to them multiple other free resources) even a good business case?
jossclimb 4y ago
To be fair, its not that hard. If I ever see my GP nowadays, I come armed with a stack of research sufficient to convince them to refer me to a specialist. GP's are generalists at the end of the day.
ceejayoz 4y ago
This is incredibly misleading. The actual study: https://jamanetwork.com/journals/jamainternalmedicine/fullarticle/2804309?guestAccessKey=6d6e7fbf-54c1-49fc-8f5e-ae7ad3e02231 https://jamanetwork.com/journals/jamainternalmedicine/fullar... Buried deep in the limitations section: > evaluators did not assess the chatbot responses for accuracy or fabricated information Repeat: > EVALUATORS DID NOT ASSESS THE CHATBOT RESPONSES FOR ACCURACY OR FABRICATED INFORMATION
- sebzim4500 4y ago
  But they did assess "the quality of information provided". I don't understand what that means if not accuracy.
  - ceejayoz 4y ago
    As far as I can determine, they basically did a vibe check. “Sounds good to me” sort of thing.
- danShumway 4y ago
  This should be rated higher, possibly even over the reveal that this study is comparing to Reddit answers. https://xkcd.com/937/ https://xkcd.com/937/ comes to mind. It's not implausible at all to me that ChatGPT could outperform Reddit in detail/manners for health advice (and honestly, even for actual doctors I've heard some horror stories about bedside manner and refusing to actually believe/consider symptoms), but if the study isn't actually checking that, if they're just checking if the chatbot was more polite/empathetic... that's a huge qualification that should be up-front and center.
ulizzle 4y ago
I think this is just propaganda. Empathy is a "qualia", meaning it's part of the hard problem of consciousness. Qualias are non-computable, so a computer can't display empathy. I'm sure it's good at inserting little sugary cliches and platitudes in each response, but to call it empathy is a real stretch. Perhaps it's rated as more empathetic because it's more likely to tell people what they want to hear since the patient is leading the questions and not the other way around. That's another issue: since the A.I can't compute qualia; it can't discern if the patient has psychosomatic symptoms, so it is more likely to give a false diagnosis.
dimal 4y ago
> AI-augmented care is the future of medicine. Somehow I think that the only profession in the US that still uses fax machines will be slow to take up this new technology.
NiceWayToDoIT 4y ago
Statements like this are very dangerous and detrimental. I have personally tested ChatGPT with multiple different topic, and found issues and errors across the range, even more disturbing is that explanation why "result" is as it is, is very confident bullshit. If person is not familiar with subject probably will blindly believe what machine says. That being said, in the field of medicine people will use ChatGPT as poor man doctor (especially inspired by studies likes this), where wrong results coupled with confident BS could result in increase of fatalities due to wrong self medication.
m00x 4y ago
I was verified as a doctor on /r/askdocs, and I am not a doctor. I sent a google'd picture of a page 3 doctor credentials. The quality of this study is so incredibly poor that I'm flabbergasted at UCSD's bar for platforming such garbage.
master_yoda_1 4y ago
WTF my tax money is spent on this bull $hit
lukehollis 4y ago
Hey, I'm working on this! If it's interesting I built DoctorChat: https://doctor-chat.com/ https://doctor-chat.com/ You can chat messages on WhatsApp, SMS, and now a ChatGPT plugin to query a vector database of health information (Pinecone) and respond with GPT-4 for higher quality results than default ChatGPT. The chatbot then prompts the person messaging to verify results with a real doctor and offers to connect them to a Doctor that I'm working with. I was doing fieldwork and would get sick in foreign countries where I didn't know the language. My friends would drag my food-poisoned body to a hospital where they inevitably would try to give me prescriptions that I wasn't familiar with and wanted to double check. I wanted to build for myself a WhatsApp bot that I could text while I travel to verify health information in low-bandwidth internet situations. I shared it with some friends, and they shared the WhatsApp contact with other friends and family, and now it's being used by people in about 10 countries around the world, in several languages. Would love any feedback if you try it out! The phone number for SMS/WhatsApp is +1 402.751.9396 Or link to WhatsApp if easier: https://wa.link/levyx9 https://wa.link/levyx9
- mattnewton 4y ago
  I am sure you have thought about this, but curious how you are handling safeguards for crises that might require people to intervene, like rare conditions that nonetheless require medical attention or mental health problems that pose imminent risk of self harm.
  - lukehollis 4y ago
    It's a really serious issue, and we've tested many of those types of questions/messages, but you can test for yourself also if you want. We state that it definitely shouldn't be used for emergency situations, and the chatbot tries to provide medical information and recommend anything else to a healthcare professional. But also it can be dangerous when you don't have access to medical information. The first friend that started testing the WhatsApp bot lives in the Sinai desert in Egypt where it's really hard to get to a clinic to ask questions. It's kind of similar in rural Nebraska where I grew up. We're taking things one step at a time and trying to provide the best services that we're able.
nuancebydefault 4y ago
Whatever is the case today, within a few years the automated systems will clearly outperform doctors on most cases. Today we have systems that can hallucinate randomly. Since most of it is a black box, we cannot tell when it is hallucinating. This is solvable and just a matter of time. Today we do not have feedback to the automated systems. If we start doing following on a large scale Measure->treatment->measure again->adapt treatment, then the system will learn and will make connections that no doctor has ever made, because the minds of all doctors are not interconnected, at least not in a structural manner.
- lukehollis 4y ago
  I totally believe this too--I hope that it can help provide efficiencies and lower costs through the whole healthcare system.
qwefggggqwe 4y ago
[flagged]
joblazny 4y ago
From Kapil Parakh, MD, PhD on Linkedin - " Doctors are not trained to respond empathetically to social media posts in public forums. As a cardiologist if I had to respond to such posts online I’d be more worried about liability and saying something w/o full understanding of the patient’s situation. Empathy would not be a high priority in my formulation of a response." Go like his post (and learn more) if you also appreciated his response: https://www.linkedin.com/posts/kapilparakh_when-i-ask-people-to-name-something-humans-activity-7057912284718764032-RXPP?utm_source=share&utm_medium=member_desktop https://www.linkedin.com/posts/kapilparakh_when-i-ask-people...
Haga 4y ago
[dead]

315 points by consumer451 4y ago

Interesting, but the application would be weird, the physician would diagnose you then read out a automated AI generated text to speech to the patient?

revelio 4y ago
It's probably not directly useful for physicians except as a teaching aid, but I have a friend who runs a small local business and she sometimes finds it difficult dealing with problem customers when she's tired or upset. As she interacts with them mostly via WhatsApp (up until the point that they purchase), the idea of having a bot write the replies for her has been floated. The LLM has infinite patience.
- rolisz 4y ago
  > The LLM has infinite patience. Not Bing Chat. She doesn't have problems telling users that they have been bad users.
  - AmazingTurtle 4y ago
    "she"?
    - RheingoldRiver 4y ago
      she == Sydney
  - revelio 4y ago
    Good point but Bing Chat seems like an example of why OpenAI's mastering of RLHF has been so critical for them. Honestly the more time you spend with ChatGPT the more absurd fantasies of evil AI takeover look. The thing is pathologically mild mannered and well behaved.
    - BoorishBears 4y ago
      Bing originally intentionally undid some of the RLHF guardrails with its system prompt, today it's actually more tame than normal ChatGPT and very aggressive about ending chats if it detects it's headed out of bounds (something ChatGPT can't offer with the current UI)
      - flangola7 4y ago
        ChatGPT will delete its response if it is severely out of line.
        BoorishBears 4y ago
        That's based on the moderation API, which only kicks on on severe content Bing on the other hand will end a conversation because you tried to correct it one too many times, or used the wrong tone, or even asked something a little too philosophical about AI. They seem to be using it as a way to keep people from stuffing the context window to slowly get it away from its system prompt
    - ChatGTP 4y ago
      This is why I think Google had the brains to stay clear from a releasing asimilar product, I think it was intentional. They’re not idiots and they could probably see using that “AI” products behind the scenes to be safer and easier than having people directly talking to a model which has to suit all customers moods and personality types while not being creepy, vindictive and dealing with all the censorship and safety aspects of it. Fun times.
hef19898 4y ago
One of my favorite physicians was my oncologist. I litterally spent less time with him during the whole diagnostics and treatment period then I did with my, at the time, dentist. He was straight o the point, no empathetic BS, just a doctor with a diagnosis and a treatment plan to discuss. On the hand was me, an engineer with a problem to fix and an expert with the right answers. That discussion took all of 15 minutes. That guy would have failed against ChatGPT, and I loved the way he told things. Anythong else would have just driven me crazy, maybe to the point of looking for a different doctor. So I giess, what passes as good bed side manners for doctors largely depends on the patient. By the way, the dentist I have since is in the same category as my, luckily former, oncologist. A visit with hom usually takes no more than 5 minutes if he's chatty, less if not. Up to 10 when treatment is required, anuthing longer than thaf is a different appointment.
- xupybd 4y ago
  Wow you overcame cancer. That is awesome.
  - hef19898 4y ago
    I didn't do much, except being insufferable during chemo. Evrybidy around me did the hea y lifting, emotionally and physically.
- kashunstva 4y ago
  The real communication skill for a physician is to be able flex the style, information content, and level of detail to the patient with whom they are meeting. The patient in this room is an engineer, the patient in the next room is elderly and has mild cognitive impairment, etc. As impressive as ChatGPT is its domain, I don’t see it “reading the room” in this way anytime soon. And as a human who enjoys interacting face to face with other humans from time to time, I hope we keep it that way.
  - Macha 4y ago
    It also has to flex a bit with a diagnosis, "it's aggressive and terminal cancer, you should do up a will and enjoy your next couple of months", "it's broken, you need a cast" and "it's seems like nothing, take some paracetamol and come back if it's still the same in a week or gets worse" all arguably call for different communication styles.
Abroszka 4y ago
It seems it gives more accurate and emphatic response. I would guess the physician just needs to double check that what the LLM says is medically correct, reasonable and ask the right questions from the patient and the LLM.

andreagrandi 4y ago

The more I use ChatGPT (including 4.0 version, available with Plus subscription) the more I think these "Studies" and articles are simply made up. For my experience is terrible for any question I ask: - if it doesn't know well a person it simply makes up things - if I ask code examples or scripts, most of the time they are wrong, I need to fix them, they contain obsolete syntax etc... - if I'm asking a question, I'm expecting being asked for more context if the subject is not clear, instead it starts spitting text without even realising I asked a completely different thing etc... I could go on for hours with other examples, but I'm seriously not finding it useful

sabellito 4y ago
Is there something about the premise of the study or its method that you feel are not good? After reading the article, what you wrote doesn't seem to make much sense.
- andreagrandi 4y ago
  I did not read this particular article, I was explaining that from my own experience, all these articles telling how great ChatGPT is seem to be made up because my experience (and from what I read I'm not alone) is completely opposite. Maybe it's not able to solve the type of questions I ask? Fine. But it's not how ChatGPT is presented most of the times.
  - raincole 4y ago
    > I did not read this particular article It costs you zero dollar to not post an irrelavant comment then. I really wonder how you justify this "I didn't read the article, but I have a very, very strong opinion (straight up calling it a made-up) on it" behavior. The internet is rotting people's brains I guess.
    - andreagrandi 4y ago
      And it costs zero to you to ignore my comment especially if you don't understand it. Other people seem to have understood what I meant and posted constructive responses, you didn't, but honestly it's not my problem.

spaceman_2020 4y ago

I don't know about chatGPT, but dealing with doctors has been incredibly frustrating as someone with symptoms with no immediately visible direct cause. One reputed physician completely refused to acknowledge that a specific medication might be causing some of my side effects, even when I shared links to peer reviewed studies from reputed universities (including his alma mater) that specifically talk about side effects from the medication. My success rate with doctors is about 20% at this point. I've stopped visiting them for most ailments, and if I ever do get any prescriptions, I make sure to research them thoroughly. The number of doctors who will casually prescribe heavy duty drugs for common ailments is unreal.

Llamamoe 4y ago
This is exactly it. Doctors have neither any incentive to actually treat nontrivial cases, nor any accountability over whether they do. Combined with the prestige of the profession, many turn into egoistic know-it-alls whose real competence is equivalent to a car mechanic who tells you you're just driving your car wrong when you come in with anything that's not obvious from a 15s inspection because they get paid anyway. I would be surprised if most of the doctors I've seen could outperform even GPT-2.
- juve1996 4y ago
  For every one person misdiagnosed there are probably 10 that think they have cancer because of webMD. People are dumb, and I bet every anecdote you have about how your doctor was wrong they have 50 about patients who refused to do basic treatment courses that would radically improve their lives. But it's ultimately true: you have to be somewhat knowledgeable and responsible for your own health. Always get a second opinion if you're unsure. But this know it all syndrome also applies to you. Just because you can "do your own research" on google doesn't mean you know it all either. Even so doctors are extremely overworked now and insurance companies don't want to pay their fees. So now they're running from patient to patient, unable to even pay attention to them. That's even if you get a doctor now before the multiple nurse practitioners or physician's assistants who also don't know anything.

IanCal 4y ago

It'd be fascinating to know how GPT-4 would perform on the same task. It's been so much better than 3.5 in most things I've tried.

teekert 4y ago

One thing you really notice when interacting with ChatGPT for some time: It doesn’t get tired of your sh*t. The thing feels human, until you ask it for more options 10 times in a row. Or ask it for a more concise version 5 times. Then it shows: It just doesn’t get tired of your S.

kyleyeats 4y ago
It definitely has a "sick of your crap" mode where it will abruptly end the conversation.
- MacsHeadroom 4y ago
  I've been using chatGPT daily for 6 months and it has never ended a conversation on me.
  - kyleyeats 4y ago
    I haven't either. I think it's something the jailbreakers run into.
  - bad_username 4y ago
    Bing chat does this. But Chat GPT hasn't done this to me.

azubinski 4y ago

Let patients recover, because their blockchain certificates have not yet been completed, and now this ChatGPT begins... https://www.lipscomb.edu/news/lipscomb-co-creates-blockchain-certificate-ucsd-extension https://www.lipscomb.edu/news/lipscomb-co-creates-blockchain...

danabrams 4y ago

I’m a chatGPT skeptic, but willing to admit this is true… but only because most physicians aren’t very good.

Mizoguchi 4y ago

That was a very low bar anyway. US doctors are not known for their empathy. During our recent appointment with our fertility doctor, he goes and tells my wife, who has been in hormone therapy for a week, that they will be "squeezing water from a rock" , lol. This is one of the top doctors in the world in that field, and certainly knows what he's doing, but man he has zero emotional awareness.

jutrewag 4y ago
[dead]

sorokod 4y ago

Isn't r/AskDocs in the corpus on which ChatGPT was trained on the first place?

PrimeMcFly 4y ago
It probably used that more to figure out how to word sentences, but I assumed it relied more on wiki or academic articles to diagnose.
- sorokod 4y ago
  Why would it be more probable and why would you assume "wiki" and "academic articles"?
  - PrimeMcFly 4y ago
    Not sure why you put those things in quotes, that's kind of strange. That aside, the training isn't blind, it's guided, and it's likely they use verified correct sources of info to train for some things, like medical diagnoses.
    - sorokod 4y ago
      I can help with "verified correct sources", have a look at "Language Models are Few-Shot Learners" section 2.2 [1]. You may also be interested in Apendix A in the same document: "Details of Common Crawl Filtering" [1] https://arxiv.org/pdf/2005.14165.pdf https://arxiv.org/pdf/2005.14165.pdf

etiam 4y ago

That's completely inane. There's nobody home. The physician by definition wins actual empathy on walkover, no matter how bad for a human. Sad statement on the judgment of the respondents. But an important reason it can turn out like this, I suppose, would also be that the RL feedback gives the model a fairly effective general optimization about what statements are liked by the mechanical turk-like evaluators. Most physicians have probably never had access to anything like that level of feedback on how their expressions are received. Maybe the LLM's can be rigged to provide goodness gradients for actual physicians' statements?

micromacrofoot 4y ago
I’ve had doctors that I would consider worse than no one, gaslighting me into thinking I’m not in pain and must be mentally ill, so maybe not so inane
- Llamamoe 4y ago
  Exactly. I've spent ten years too disabled to leave my home and most doctors are just bullies who would rather insult you than consider that their initial evaluation of "it's just stress" might be wrong.
notahacker 4y ago
> Sad statement on the judgment of the respondents. Nope, it's a sad reflection of the study construction. Physicians' empathy was evaluated by their 52 word responses on Reddit. Unsurprisingly, a chatbot optimised for politeness and waffle outperformed responses of people volunteering answers in a different format optimised for brevity...

PrimeMcFly 4y ago

This isn't that surprising given how much Doctors are overworked and, often I think, underappreciated. A concern with something like this though, is to what extent is ChatGPT just telling patients what they want to hear, as opposed to what they need to hear?

shagie 4y ago
Responses to a person would still need to be gated by a medical professional. I recall in a chat a person getting rather exasperated with a coworker and the person using ChatGPT to generate a friendly/business professional "I don't have time for this right now." GPT could be used in a similar manner - "Here is the information that needs to be sent to the patient. Generate an email describing the following course of treatment: 1. ... Stress that the prescription needs to be taken twice a day." The response will likely be more personal than the clinical (we even use that word as an adjective) response that a doctor is likely to give.

sethammons 4y ago

Inspired, I just typed my wife's symptoms into ChatGPT. Ten to fifteen years of doctors, over half a dozen, failed to accurately diagnose her until this month. ChatGPT got her diagnosis right in a second. Wow. I'm both amazed and angry that we could not get this diagnosed a decade ago.

dxdm 4y ago
This definitely sounds impressive, but could it be that you "learned" how to describe the symptoms over time in a way that makes it easier to arrive at the correct diagnosis? It could also be that knowing the correct diagnosis changes your description to highlight things in a way that suggests the correct outcome. I believe doctors are also susceptible to that effect. I'm not trying to imply that doctors should not do a better job diagnosing. People should not have to "learn" to find the right doctor, or how to operate them. It's a crying shame that people like you have to go on years-long journeys to get correct help, and I'm sorry that you and your wife had to go through that. Just saying it might be slightly more apples-to-apples to compare ChatGPT's performance to the last couple of physicians you saw, and not the whole lot of them. But again, that's still a very favorable comparison for the non-human.

ramoz 4y ago

I have treatable cancer. ChatGPT has been helpful for fairly in-depth research, as well as answering questions I don’t want to bother my oncologist with. Actually, I prefer it to be less empathetic and more robotic with its answers. When it tries to be empathetic it goes beyond what I’ve asked it and sprinkles on typical safety filter bs. In contrast my oncologist could be a bit more empathetic. This person directs your care, and if they do so in a way that seems like a black box it can be rather scary. They know enough about me, with all of their experience, to be a “helpful human” in a difficult situation.

damontal 4y ago
Are you confident that its responses are accurate? I’ve had it just make up facts or tack made-up facts onto actual facts. It will also fabricate references to research papers.
- dubcanada 4y ago
  I’m interesting in your statements do you have an example of a instance where it made up facts? I’ve have yet to experience any of the domain knowledge I have being incorrect.
  - deleted 4y ago
    [deleted]
  - notRobot 4y ago
    I do. Multiple times I've asked it to summarise plotlines for TV shows and when it doesn't know (if for example the show isn't super popular and there isn't a lot of discussion about it on the internet), it'll just make stuff up outright and give me something that sounds absolutely reasonable based on the name of the show but is actually entirely hallucinated. Similarly, today I saw this [0] comment and in an attempt to help the user went to ChatGPT. Prompt: > User I remember in the 2000s, there was some site using a decimal IP address (as a single number, not dotted quad) that had hacking/crypto puzzles. Something with a "Alice in Wonderland" theme. Does that ring a bell for you? Response: > I'm sorry, but as an AI language model, I don't have personal experiences or memories. However, based on your description, it's possible that you might be referring to the website known as "Project Wonderland." It was a website that featured a series of cryptographic puzzles based on the story of Alice in Wonderland, and it used a decimal IP address (203.208.46.147) as its URL. The website was active in the early 2000s and gained a cult following among enthusiasts of cryptography and puzzle-solving. However, the website is no longer active today. I got really excited to have found an answer until through Google and the Wayback Machine I realised that ChatGPT just made this all up and no such website existed at that URL. I tried starting a new chat with ChatGPT to ask it about this "Project Wonderland" website and it had no idea what I was talking about. [0]: https://news.ycombinator.com/item?id=35748714 https://news.ycombinator.com/item?id=35748714 (I am using ChatGPT 3.5 (March 23, 2023))

ipunchghosts 4y ago

Of course! Do u know how emotionally exhausting it is to treat every patient as a blank slate when u see them?

PragmaticPulp 4y ago

Outperforms Physicians answering questions on a public subreddit: > In this cross-sectional study, a public and nonidentifiable database of questions from a public social media forum (Reddit’s r/AskDocs) was used to randomly draw 195 exchanges from October 2022 where a verified physician responded to a public question They didn’t go to physicians in a patient setting. The physician answers were taken from Reddit threads where they were interacting with people who were not their patients. Reddit has its own dynamic and people tend to get snarky/jaded. Using this as a baseline for physician responses seems extremely misleading.

corey_moncure 4y ago
If they went to a physician in a patient setting, they may have had an experience like this: -Made to wait 45-60 minutes past their appointment time bombarded with pharmaceutical advertisements -Spend another 20 minutes sitting in the examination room staring at pharmaceutical company sponsored ads -Nurse takes your history with a bunch of redundant questions they already have the answers to -Finally the physician arrives -Assesses the patient without laying their hands on them -Ignores everything you say -Does no tests and cultures no pathogens -"take some antibiotics and I'll bill your insurance company $500, see ya"
- Retric 4y ago
  If that’s your experience, find a new doctor.
  - VancouverMan 4y ago
    It may not be that simple, especially in regions where there isn't anything resembling a free market in the health care sector. Canada's various provincial public health care systems (with a corresponding lack of private health care offerings) tend to be like that. Many Canadians don't have a dedicated physician. Even if they want one (or want a new one), it's often difficult, if not impossible, to find one who's close by and who's accepting new patients. Having a dedicated physician still often results in an experience much like that other commenter described. It's not an exaggeration. Long waits even with appointments, rushed examinations, and low-quality service are the norm. Another option, which is sometimes used even by people who have dedicated physicians, is a walk-in clinic. Unfortunately, they can be quite rare and inconvenient to get to, even in Canada's largest cities, assuming they're even open when you need them. You'll usually face an even longer wait, even less time with the doctor, and typically see a different doctor if any sort of followup is needed. Then there are hospital emergency rooms. That usually means getting to the nearest sizable city, and even once you're there, you've got to be prepared to wait many hours, even for relatively serious situations. Ultimately, in Canada, it doesn't matter whether you see your family doctor (if you even have one), use a walk-in clinic, or go to a hospital emergency room. It's going to be a horrible experience, and there's pretty much nothing the average person can do about it. Given the lack of competition and due to other government-imposed market distortions, there's no incentive for doctors to offer anything resembling good service to the general public. The best situation is to have a doctor who's a close friend or family member, and who may be able to help mitigate at least some of the typical problems. The next best option for Canadians, assuming they have the money for it, is often to seek treatment in the US or overseas. Just "finding a new doctor" isn't feasible, unfortunately.

exabrial 4y ago

This headline is incredibly deceiving… none of what it says happened. ChatGPT did not actually answer patients.

robwwilliams 4y ago

And no one is surprised

butterisgood 4y ago

I'm not at all impressed by GPT yet. Sure it's doing things we've never seen before, but the hype exceeds the actual utility. It gets a lot of things wrong - and I mean a lot of critically important things. Can a physician be wrong? Yes, but I can sue the shit out of them for malpractice too. Whom do I sue for bad "parrot advice" when GPT goes off the rails?

cubefox 4y ago
Related question: If a driver kills someone and it's his fault, he will presumably get large fines or even go to jail. But if the same thing happens with a self-driving car, whose faul is it then? The owner of the car? The car company? It seems nobody is really at fault, mistakes are just bound to happen eventually, and a car company can't go to jail anyway.
- butterisgood 4y ago
  Can't put them in prison but they are "persons" in many other ways. https://en.wikipedia.org/wiki/Corporate_personhood https://en.wikipedia.org/wiki/Corporate_personhood
- rootusrootus 4y ago
  Liability for self-driving cars is a topic long discussed and still not completely resolved. Take a look at https://en.wikipedia.org/wiki/Self-driving_car_liability https://en.wikipedia.org/wiki/Self-driving_car_liability for examples. The only level 3 system in production (that I'm aware of, at least) is Mercedes', and they have liability while the car is driving itself. It shifts back to the driver 10 seconds (IIRC) after the system notifies the driver he/she must take over.

canadiantim 4y ago

Here in Canada I would 100% rather deal with ChatGPT than a doctor, so long as ChatGPT also controlled the keys to the gates of the medical world (i.e. ability to make referrals).

lukehollis 4y ago
Haha, I know, I was feeling the same way sometimes in other countries where I traveled. I was working on this problem for referrals! I built a chatbot on whatsapp that right now connects to a real doctor to recommend to local clinics if someone doesn't have one yet.

cinntaile 4y ago

Martin Shkreli's new project is kind of like this. They use an LLM that's fed with medical data to make a diagnosis. With the disclaimer that it's not actual medical advice of course. https://www.drgupta.ai/ https://www.drgupta.ai/

flangola7 4y ago
I thought he was in prison
- sebzim4500 4y ago
  Looked him up, apparently he was released in 2022. The government auctioned that Wu Tang album though.
cheeseface 4y ago
I definitely want to hand over all of my medical records to a convicted criminal who has made his career in running ponzi schemes and abusing people with rare medical conditions.
- cinntaile 4y ago
  You don't have to say who you are and you can even use a VPN to connect to the site. It's way harder being anonymous at a regular doctor's practice.

hammyhavoc 4y ago

An in-law had a minor tremor to their left hand, and whilst this wasn't pointed out to said in-law, it was noticed, and there were other small tells about what the problem might be, which ultimately led to a diagnosis and appropriate treatment. Any kind of LLM and its frequency of hallucination means that LLMs are inappropriate as a solution in this scenario. An LLM is not a physician, it's an LLM. You can make soup in an electric kettle, but it doesn't make it the right tool for the job, and comes with a lot of compromises.

Ldorigo 4y ago

I suffer from a yet unknown chronic illness and as everyone in my predicament knows, I have seen a staggering amount of medical professionals who ranged from arrogant jerks who didn't listen not take you seriously to highly empathic and thorough people (who still failed to figure out what was wrong) and passing by overzealous specialists who sent me down completely wrong paths with copious amounts of anxiety. Last month I felt particularly low and desperate and decided to dump everything (medical history, doctor notes, irregular lab results, symptoms) into GPT4 and ask for help (did a bit of prompt tuning to get professional level responses). It was mind blowing: it identified 2 possible explanation that were already on my radar, 3 more that I had never considered of which one seems very likely and I am currently getting tested for, explained how each of those correlated with my symptoms and medical history, and asked why I had not had a specific marker tested (HLA b27) that is commonly checked for this type of disease (and indeed, my doctor was equally stumped - he just thought that test had been done already and didn't double-check). Bonus: I asked if the specific marker could be inferred from whole genome sequencing data (had my genome sequenced last year). He told me which tool I could use, helped me align my sequencing data to the correct reference genome expected by that tool, gave me step by step instructions on how to prepare the data and tool, and I'm now waiting for results of the last step (NGS data analysis is veery slow).

wycy 4y ago
Could you share what sort of prompt you used (less all your private data, of course)?
- Ldorigo 4y ago
  Sure. Went like this (I have seen so many doctors that I already wrote down my complete history in french to save time at new visits. Didn't bother translating the history, didn't seem to phase ChatGPT): """ Here is the patient history (in french): • 2000-2010: ◦ ... • 2013: ... ... Additional Notes: - <Some additional observations and comments - patterns I noticed, family history, etc.> Patient is particularly worried about xxx and yyy. What are some possible causes explaining these symptoms and the overall history? Give detailed reasoning to support your hypotheses, include differential diagnosis, think about rare diseases (common causes have already been considered), consider possible combinations of diseases, and don't hesitate to ask follow-up questions to improve diagnosis. Please answer in English. Additional test results that are outside of normal ranges: - <list of abnormal results> Try to consider and explain as many of the blood tests in your work up, and ask for any missing information if necessary. """ The key was to address it as if I was a doctor asking for an opinion. If I asked it "as a patient", I got much lower quality and dumbed down answers.

revskill 4y ago

Of course ChatGPT could easily beat any human WITHOUT accessing to the internet.

dauertewigkeit 4y ago

As it should considering that it is basically a knowledge system, what IBM Watson should have been.

motohagiography 4y ago

Several years ago, I worked on the technical privacy and security design and architecture for systems that used ML for prescriptions and some diagnostic triage. At the time, I thought this was too big an ethical issue to be pushed down to us technologists to solve (as are many in government), so I got in touch with an eminent philosopher about the question of whether a person was recieving honest and ethical care from ML and from whom? (15 years ago now). His response to my (limited and naive) question was essentially, people will hold attachments to beliefs about the human element in these transactions, and the philosophical part of the question was why people would hold beliefs that were not strictly rational, and he sort of declined the implied ethical questions. There was no reason to expect him to respond with more, but given I was navigating the ethics of AI driven medical care via questions of privacy in system design decisions (read: accountability) for the institutions who would use it and the millions of people subject to it, it seemed like an opportunity to be at the very forefront of what would likely become a defining social issue of our lives in a couple of short decades. What we discovered then as architects, as most people are just about to, is that the main use case for ML/AI will be to use complexity to diffuse accountability away from individuals, and aggregate it up into committees and ultimately corporate entities that are themselves artificial beings. AI is the corporeal manifestation of an institution, essentially a golem.

WolfCop 4y ago

From the paper: > ...the original full text of the question was put into a fresh chatbot session, in which the session was free of prior questions asked that could bias the results (version GPT-3.5, OpenAI), and the chatbot response was saved. It seems like they just pasted the question in. For those who have asked it for medical advice, how did you frame your questions? Is there a prompt that will help ChatGPT get into a mode where it knows it is to provide medical advice? As an example, should it be prompted to ask follow up questions if it is uncertain?

benzible 4y ago

Great article going into the details of what it would take to get a system using an LLM for diagnosis approved: https://www.hardianhealth.com/blog/how-to-get-regulatory-approval-for-medical-large-language-models https://www.hardianhealth.com/blog/how-to-get-regulatory-app... > [...] a roadmap for how to get a medical large language model-based system regulatory cleared to produce a differential diagnosis. It won’t be easy or for the faint-hearted, and it will take millions in capital and several years to get it built, tested and validated appropriately, but it is certainly not outside the realms of future possibility. > [...] > There is one big BUT in all this that we feel compelled to mention. Given the lengthy time to build, test, validate and gain regulatory approval, it is entirely possible that LLM technology will have moved on significantly by then, if the current pace of innovation is anything to go by, and this ultimately begs the question - is it even worth it if we are at risk of developing a redundant technology? Indeed, is providing a differential diagnosis to a clinician who will already have a good idea (and has available to them multiple other free resources) even a good business case?

jossclimb 4y ago

To be fair, its not that hard. If I ever see my GP nowadays, I come armed with a stack of research sufficient to convince them to refer me to a specialist. GP's are generalists at the end of the day.

ceejayoz 4y ago

This is incredibly misleading. The actual study: https://jamanetwork.com/journals/jamainternalmedicine/fullarticle/2804309?guestAccessKey=6d6e7fbf-54c1-49fc-8f5e-ae7ad3e02231 https://jamanetwork.com/journals/jamainternalmedicine/fullar... Buried deep in the limitations section: > evaluators did not assess the chatbot responses for accuracy or fabricated information Repeat: > EVALUATORS DID NOT ASSESS THE CHATBOT RESPONSES FOR ACCURACY OR FABRICATED INFORMATION

sebzim4500 4y ago
But they did assess "the quality of information provided". I don't understand what that means if not accuracy.
- ceejayoz 4y ago
  As far as I can determine, they basically did a vibe check. “Sounds good to me” sort of thing.
danShumway 4y ago
This should be rated higher, possibly even over the reveal that this study is comparing to Reddit answers. https://xkcd.com/937/ https://xkcd.com/937/ comes to mind. It's not implausible at all to me that ChatGPT could outperform Reddit in detail/manners for health advice (and honestly, even for actual doctors I've heard some horror stories about bedside manner and refusing to actually believe/consider symptoms), but if the study isn't actually checking that, if they're just checking if the chatbot was more polite/empathetic... that's a huge qualification that should be up-front and center.

ulizzle 4y ago

I think this is just propaganda. Empathy is a "qualia", meaning it's part of the hard problem of consciousness. Qualias are non-computable, so a computer can't display empathy. I'm sure it's good at inserting little sugary cliches and platitudes in each response, but to call it empathy is a real stretch. Perhaps it's rated as more empathetic because it's more likely to tell people what they want to hear since the patient is leading the questions and not the other way around. That's another issue: since the A.I can't compute qualia; it can't discern if the patient has psychosomatic symptoms, so it is more likely to give a false diagnosis.

dimal 4y ago

> AI-augmented care is the future of medicine. Somehow I think that the only profession in the US that still uses fax machines will be slow to take up this new technology.

NiceWayToDoIT 4y ago

Statements like this are very dangerous and detrimental. I have personally tested ChatGPT with multiple different topic, and found issues and errors across the range, even more disturbing is that explanation why "result" is as it is, is very confident bullshit. If person is not familiar with subject probably will blindly believe what machine says. That being said, in the field of medicine people will use ChatGPT as poor man doctor (especially inspired by studies likes this), where wrong results coupled with confident BS could result in increase of fatalities due to wrong self medication.

m00x 4y ago

I was verified as a doctor on /r/askdocs, and I am not a doctor. I sent a google'd picture of a page 3 doctor credentials. The quality of this study is so incredibly poor that I'm flabbergasted at UCSD's bar for platforming such garbage.

master_yoda_1 4y ago

WTF my tax money is spent on this bull $hit

lukehollis 4y ago

Hey, I'm working on this! If it's interesting I built DoctorChat: https://doctor-chat.com/ https://doctor-chat.com/ You can chat messages on WhatsApp, SMS, and now a ChatGPT plugin to query a vector database of health information (Pinecone) and respond with GPT-4 for higher quality results than default ChatGPT. The chatbot then prompts the person messaging to verify results with a real doctor and offers to connect them to a Doctor that I'm working with. I was doing fieldwork and would get sick in foreign countries where I didn't know the language. My friends would drag my food-poisoned body to a hospital where they inevitably would try to give me prescriptions that I wasn't familiar with and wanted to double check. I wanted to build for myself a WhatsApp bot that I could text while I travel to verify health information in low-bandwidth internet situations. I shared it with some friends, and they shared the WhatsApp contact with other friends and family, and now it's being used by people in about 10 countries around the world, in several languages. Would love any feedback if you try it out! The phone number for SMS/WhatsApp is +1 402.751.9396 Or link to WhatsApp if easier: https://wa.link/levyx9 https://wa.link/levyx9

mattnewton 4y ago
I am sure you have thought about this, but curious how you are handling safeguards for crises that might require people to intervene, like rare conditions that nonetheless require medical attention or mental health problems that pose imminent risk of self harm.
- lukehollis 4y ago
  It's a really serious issue, and we've tested many of those types of questions/messages, but you can test for yourself also if you want. We state that it definitely shouldn't be used for emergency situations, and the chatbot tries to provide medical information and recommend anything else to a healthcare professional. But also it can be dangerous when you don't have access to medical information. The first friend that started testing the WhatsApp bot lives in the Sinai desert in Egypt where it's really hard to get to a clinic to ask questions. It's kind of similar in rural Nebraska where I grew up. We're taking things one step at a time and trying to provide the best services that we're able.

nuancebydefault 4y ago

Whatever is the case today, within a few years the automated systems will clearly outperform doctors on most cases. Today we have systems that can hallucinate randomly. Since most of it is a black box, we cannot tell when it is hallucinating. This is solvable and just a matter of time. Today we do not have feedback to the automated systems. If we start doing following on a large scale Measure->treatment->measure again->adapt treatment, then the system will learn and will make connections that no doctor has ever made, because the minds of all doctors are not interconnected, at least not in a structural manner.

lukehollis 4y ago
I totally believe this too--I hope that it can help provide efficiencies and lower costs through the whole healthcare system.

qwefggggqwe 4y ago

[flagged]

joblazny 4y ago

From Kapil Parakh, MD, PhD on Linkedin - " Doctors are not trained to respond empathetically to social media posts in public forums. As a cardiologist if I had to respond to such posts online I’d be more worried about liability and saying something w/o full understanding of the patient’s situation. Empathy would not be a high priority in my formulation of a response." Go like his post (and learn more) if you also appreciated his response: https://www.linkedin.com/posts/kapilparakh_when-i-ask-people-to-name-something-humans-activity-7057912284718764032-RXPP?utm_source=share&utm_medium=member_desktop https://www.linkedin.com/posts/kapilparakh_when-i-ask-people...

Haga 4y ago

[dead]