58 ms·

I’m an ER doctor. Here’s how I’m already using ChatGPT to help treat patients

275 points by SLHamlet 3y ago

bemusedthrow75 3y ago
Two shocking things about this article: The first: with his doctor hat on, he should understand that the value is the piece of paper and reading from it consistently to relatives who may have mild cognitive impairment. ChatGPT wrote essentially identical content to the words he'd been delivering himself. The professionally responsible thing to do would have been to type up or get someone to type up his _actual words_, print those out, and sign his name to them. He is -- for marketing hype cycle reasons -- projecting magic onto AI and ChatGPT when anyone who has ever known anyone close with mild cognitive impairment knows that the consistent, calmly repeated message is the magic. This is really irresponsible at worst, and empty, AI-hopey-changey nonsense at best. The second: everyone here suggesting that this is a valid or appropriate or even appealing use of AI. Look: if you can't do empathy (and not all of us can), hire an actual person who can do empathy! Make that their job, support them in it, celebrate their ability to do it and reward them properly. They are worthwhile people do who important jobs. Don't give the job to an AI to produce a fake veneer of empathy. If you can't understand why faking and automating empathy is not a substitute for actually trying to find real empathy in a role where someone could be an effective emotional bridge between two worlds, and if you can't see that this kind of bland/BS/automated sincerity application of AI is really toxic, I don't know what to say to you.
- catchnear4321 3y ago
  the flip side of this is that the model acts as a universal translator. not just for language but also for emotional valence and context. are you a doctor? worked in an er? there are a lot of strong words about what doctors should and shouldn’t do, with regard to empathy and compassion. there is a shocking lack of compassion or empathy in the very words demanding them. (there is also a rather large assumption that the doctor would individually be able to hire someone on a whim, to talk nice to people. in an er?) the value of the paper, of the words, is that it helped bridge a recognized communication divide. the purpose? a combination of compassion and treatment. help the well-meaning elderly children understand well enough to keep them from indirectly killing their mother.
  - bemusedthrow75 3y ago
    > are you a doctor? worked in an er? No, I'm not. My dad -- who died with dementia -- was in medicine for his entire life. I am writing about what people with cognitive impairment tend to need from the perspective of reassuring, daily, a highly intelligent person who couldn't remember stuff. > there are a lot of strong words about what doctors should and shouldn’t do, with regard to empathy and compassion. there is a shocking lack of compassion or empathy in the very words demanding them. The bit about hiring someone with empathy was quite clearly in my second point, which was aimed at the HN audience, not at the doctor. (Though the guy who wrote the article isn't writing as a doctor in some medical publication, he is writing an executive at VC firm that backs AI startups, as other commenters have observed.) I have a lot of compassion for doctors. I don't have a lot of compassion or empathy for motivated reasoning in AI marketing bullshit written by well-remunerated people who are trying to force a technology into healthcare where it does not belong. And yes, the value of the piece of paper is that it bridged the divide. That's exactly what I said. It does it through consistency and constancy. ChatGPT has _no_ magical impact on that.
    - catchnear4321 3y ago
      > The bit about hiring someone with empathy was quite clearly in my second point, which was aimed at the HN audience, not at the doctor. then the assumption of ability to hire or impact hiring is broader. the article author is quite biased, yes. part of the problem with current “ai” - bias from all sides. chatgpt is not magical. it is a tool. a novel one. that many wish to sell. none of that lessens the possibilities it presents. nor the risks. but there are real, beneficial use cases. with a little time spent. not everyone wishes to put greed ahead of linear algebra, or math before compassion.
      - bemusedthrow75 3y ago
        > then the assumption of ability to hire or impact hiring is broader. It is, but so what? What I am suggesting is that using AI tools to fake empathy is worse than not being able to do empathy yourself. This should be evident. If you have a task that requires significant empathy (a patient advocate, a customer advocate, a serious complaints line, anything with extensive exposure to the emotions of your customer) and you feel unable to do that (not everyone can), the ethical, responsible thing to do is to hire someone who does that job properly. Professionalise caring. Don't just give the role to an AI that can fake it with a simulacrum of past human caring that it read on the (largely corporate) internet. We are heading down a dark path if we pretend that automated fake empathy is acceptable. Edit: and yes, people fake empathy all the time, they copy and paste it. Of course they do. But each time someone fakes empathy there is a glimmer of a chance that they will realise that it's a bad, inappropriate thing to do. There's a glimmer of a chance that they go back to their boss and say "you know, I feel bad brushing them off; what they are going through is real and awful and we should address this more personally". AI empathy will never do that. And we should feel very, very nervous about using AI to deliver highly bespoke fake empathy to eliminate any guilt from that.
Applejinx 3y ago
I'd bet money that the Medium article has also been, in part, rewritten by ChatGPT. Here's why: the story being told is that of a doctor, needing to communicate, being able to get through to 'regular folks' through talking down to them using ChatGPT (which is actually well suited to that task, once you acknowledge that hallucination can also mean losing stuff in translation). Then, the article itself, including the description of the loving family's reaction, is VERY characteristic of the way ChatGPT 'talks down'. It's hallucinatory happytalk that paints with a very broad brush, and delivers not the tone you want, but an abstracted 'popularized' tone likely to convince large numbers of stupid and gullible people. That tone's hard to miss, and it's popping out all over the article. Points for committing to the bit. However, what's pictured is clearly a level of Hell, or possibly an out-take from Idiocracy. You won't believe what happened next! :D
- Xen9 3y ago
  "...hallucinatory happytalk that paints with a very broad brush, and delivers not the tone you want, but an abstracted 'popularized' tone likely to convince large numbers of stupid and gullible people." You capture essence beautifully.
  - Applejinx 3y ago
    Only because I've written many words and had very specific concepts, 'essence', to represent with words. What I'm saying is not 'popularized', it's a set of very specific points. You could GPT-ize what I said into 'feel-good talk' or 'hallmark cards' but you would lose the concept of 'the very broad brush does not deliver the tone you want, but a generalized one that will reach more people through being already based in how they see things'. If you need to get a mass audience to see a new thing, ChatGPT is NOT your friend… it will tell them what they already know… the wild thing is how the original claim is just this, 'ChatGPT got an audience to see a new thing', but the ChatGPT-style payoff of 'they smiled bovinely and accepted the new information' is clearly what the ARTICLE AUTHOR wanted to hear, and thus it's a really suspicious take.
  - easyThrowaway 3y ago
    A quick way I've found to recognize ChatGPT-written text is upon reading it aloud, it 's kinda obvious how constant and monotone it sounds, no matter the chosen style. It gives a very "metronomic" rhythm, almost like a krautrock song. It's kinda obvious once you notice it.
isykt 3y ago
> But in these scenarios, reviewing existing work is usually much faster than starting from scratch. This is true, but it’s also a different skill than writing is. There are people who write well enough but are terrible editors. I fear for them in the generative AI workplace.
kwhitefoot 3y ago
Sounds to me as though the patient's children simply regarded the printed word as more authoritative.
stcroixx 3y ago
I just don’t believe he read that to their face and didn’t get punched. No way this happened.
scrum-treats 3y ago
I can't wait to go to the doctor, they ask me my favorite TV show/movie/video game, and then I get my diagnosis explained to me within the context of that show/movie/game. I could ask GPT to "doctor up" the explanation, and ChatGPT gives me a little more technical/medical jargon. Or just straight cascades of analogy.
- scrum-treats 3y ago
  This is sarcasm, by the way.
paulryanrogers 3y ago
> ...only if you feed it perfect information, and the actual patient has a classic presentation, which is rarely the case. (And you don’t mind a 50% success rate that often misses life-threatening conditions.) This sounds like a step backwards?
- danmaw 3y ago
  It isn't a doctor, it's trained on Reddit threads and other random shit off the internet not medical journals.
  - jprafael 3y ago
    Given that "The Pile" includes PubMed and NIH as data sources it would be unlikely to have GPT4 not use them at all. Even GPT3 uses Wikipedia which does have (mostly) factual data with cited sources.
    - ekianjo 3y ago
      Problem is half of the publications or even more is pure garbage that cannot be reproduced so there is that.
    - tazjin 3y ago
      > Even GPT3 uses Wikipedia which does have (mostly) factual data with cited sources. There's a LOT of stuff on Wikipedia where the source is a link to some random, long article and it's unclear where exactly the referred to information is coming from. Gets significantly worse for any "hot" topic.
    - kaliqt 3y ago
      It's lossy though, it tries to remember everything and relate that to everything else. Performance would increase significantly if it were tuned for that use case.
  - fnordpiglet 3y ago
    You should try asking it detailed questions about medicine. It’s fairly deep and broad in its medical “knowledge.” But it’s not good at deductive or inductive reasoning and has no agency, so is entirely unsurprising it’s not good at differential diagnosis. We actually have good differential diagnosis expert systems the challenge is getting the providers to input queries properly. I can imagine GPT4 acting as an intermediary between human natural language and expert systems to great effect. It seems rather rash to be judging these technologies based off a crappy web UI thrown on a chatbot a few months ago.
pedalpete 3y ago
Will there be a Dr House (is there a more current TV doctor) video version which the family can interact with which would make them feel like they are getting the best care in the world from a famous doctor? But also, if ChatGPT has a 50% chance of giving the wrong diagnosis, can you trust it more than 50% to give the correct reasoning to the family as to the activity of the medical staff?
- weaksauce 3y ago
  no, you still have to read it. > In each case, the output from the hungover intern/ChatGPT needs to be carefully checked before it’s used. But in these scenarios, reviewing existing work is usually much faster than starting from scratch.
- tazjin 3y ago
  > if ChatGPT has a 50% chance of giving the wrong diagnosis It depends on the average chance of the doctor giving the wrong diagnosies (I honestly have no idea?!)
  - OJFord 3y ago
    My experience is second-hand, but they seem to be pretty split between seeing exciting rare diagnoses everywhere (students, juniors primarily, or one's own particular research interest) and biased against them, it's ~never that, it's the same boring thing we saw ten of yesterday (jaded specialists primarily).
  - EA-3167 3y ago
    There are different figures, partly as a result of how this question is studied. The Mayo Clinic did a famous study on people who sought a second opinion, and they found that something like 12% of people seeking that second opinion were correctly diagnosed. Obviously this isn't a fair sample since this is not only people seeking a second opinion, but one from a prestigious source. This narrative review discusses the issue from a global perspective, concluding that most people will experience at least one misdiagnosis. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5502242/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5502242/ Article on Mayo study, with link to study: https://newsnetwork.mayoclinic.org/discussion/mayo-clinic-researchers-demonstrate-value-of-second-opinions/ https://newsnetwork.mayoclinic.org/discussion/mayo-clinic-re...
ilaksh 3y ago
Theoretically something like BioMedLM and/or a vector search over clinical research and/or GPT-4 with cited sources etc. could be useful in some diagnostic circumstances to at least give doctors ideas, right? What he is talking about here is just an incredibly common scenario that the doctor and everyone is already very familiar with. Which is basically delaying the inevitable for people who are in some ways already gone. Also my understanding is that the ChatGPT API does have some level of privacy now although maybe not compliant with the privacy laws which seems somewhat burdensome on the entire medical industry (overcompensating in some detrimental ways). It seems like the fundamental problem is aging. Maybe a leading edge LLM approach can help keep people up to speed on aging research. Although probably that's pushing it because you would need to get very deep into mitochondrial damage or whatever.
- wolverine876 3y ago
  > Theoretically something like BioMedLM and/or a vector search over clinical research and/or GPT-4 with cited sources etc. could be useful in some diagnostic circumstances to at least give doctors ideas, right? Anything is possible, but consider this doctor's review of ChatGPT's diagonistic output: > ChatGPT works fairly well as a diagnostic assistant — but only if you feed it perfect information, and the actual patient has a classic presentation, which is rarely the case. (And you don’t mind a 50% success rate that often misses life-threatening conditions.) For any new technology to be useful, it needs to do more than just work; it must be significantly more productive than the alternative.
  - ilaksh 3y ago
    That review is one anecdote which does not specify the ChatGPT model used (which vary widely in performance), is not BioMedLM, is not a vector search, etc.
Animats 3y ago
When you need blithering, ChatGPT is there for you.
- wolverine876 3y ago
  You are just in wishful denial about the technological disruption: ChatGPT will rapidly and dramatically increase productivity by displacing human workers. ChatGPT will automate 99.9% of what's on the Internet, which is blithering, displacing what most human labor is spent on these days, and leaving humans with no demand for their labor but going back to producing accurate, valuable knowledge. The halcyon days are over. Post-truth technology has followed the typical maturation cycle, and now is at the point that we can automate it. (HN is dead. This post could be written by ChatGPT or a future LLM.) ;) :DDDDDDD
- TeMPOraL 3y ago
  In other words: ChatGPT is there for you to help with most human interaction.
sfink 3y ago
Kind of fascinating that the benefit here is not so much artificial intelligence as artificial empathy.
- VWWHFSfQ 3y ago
  "here's a squeeze ball. go sit outside and squeeze it."
- tshadley 3y ago
  All cynicism aside, there's vastly more in the collective writings of humans on empathy than medicine.
  - version_five 3y ago
    See my sibling comment just posted, I don't belive that's relevant. You can always screw up facts, it's a high bar. A loosely constrained example of an empathetic speech much easier. Despite the fact that medicine deals in facts (and this would hold true for an even more black and white discipline) so all you need in principle to learn from are a few examples, while empathy is much more subtle and would require a representation to be learned from many varied examples.
    - fsckboy 3y ago
      > the fact that medicine deals in facts medicine benefits from facts, but is largely based on experienced, expert opinion
      - riwsky 3y ago
        And experience is the practical contact with and observation of facts, over an extended period. Most English dictionaries explicitly separate out “formal judgment by an expert” as a distinct meaning for “opinion” from the more common meaning of personal subjective no-facts-needed belief.
- Nition 3y ago
amstan 3y ago
> Satisfying HIPAA rules around patient privacy alone may take many years, or decades, to resolve themselves before we could even contemplate directly using programs like ChatGPT in a medical theater. This is easily solved by not using someone else's API. Here's $500 worth of hardware you can get on ebay running Vicuna-13B, locally: > IV fluids may not be the best treatment for someone with severe pulmonary edema and respiratory distress because it could make their condition worse. This is because when a person has severe pulmonary edema, their lungs are already filled with fluid, which makes it harder for them to breathe. Adding more fluid to their body could increase the pressure in their lungs and make it even harder for them to breathe. This could be life-threatening. > Instead, the best course of action would be to focus on treating the underlying cause of the pulmonary edema, such as heart failure or a lung infection. This may involve giving medications to help the heart pump more effectively or to reduce inflammation in the lungs. The patient may also need oxygen therapy to help them breathe more easily. > It's important to remember that every case is unique, and the best course of treatment will depend on the individual patient's condition. The healthcare team will do their best to provide the most appropriate care for the patient based on their specific needs.
- cced 3y ago
  How much time does this take to get produced locally?
  - smoldesu 3y ago
    I run a similar Vicuna bot on a free ARM VPS from Oracle. Inference takes ~20 seconds for ~200 tokens on the CPU, so it should stream results about half as fast as ChatGPT. ...however, that was on cheapo ARM hardware. On a hospital budget I bet you could beat OpenAI with a local model pre-mapped in GPU memory on a 3090.
    - cced 3y ago
      Do you guys have any reference material on how to get this started?
deleted 3y ago
[deleted]
- deleted 3y ago
  [deleted]
CodeWriter23 3y ago
I’d say doc should be expecting a visit from the FDA shortly about his unlicensed medical device.
- ftxbro 3y ago
  maybe the FDA is busy visiting every doc that uses google
  - CodeWriter23 3y ago
    Are you suggesting ChatGPT is merely an overhyped search engine?
    - ftxbro 3y ago
      no, you said that. i'm saying that if gpt is a medical device then so is google, and doctors use google all the time
- captainoats 3y ago
  The FDA regulates bringing a device to market commercially, doctors can use regular COTS software.
peter_l_downs 3y ago
This is the most “that didn’t happen” story I’ve seen in a while. No way the concerned family responded positively to the doctor reading a script to them off a clipboard, forgot about it, and then responded positively to other nurses/doctors reading the same script to them again.
- bagels 3y ago
  My wife works in cardiac and sees similarly confused patients and elderly family members. Have you tried to have conversations with people experiencing dementia?
  - peter_l_downs 3y ago
    Yes, it is unfortunate and difficult and I did not realize reading a prepared script off a clipboard would have helped me get my point across better. I could only see it causing them to view me as stonewalling them or cause them further confusion. I may have to try this for myself though, I am not a doctor and I haven’t tried it before.
  - epcoa 3y ago
    Yes, many times. Certainly don't do it by reading off a preprinted script.
- talldatethrow 3y ago
  You've clearly never worked in sales. Sometimes you simply can't make progress and another person comes in saying the exact same thing and the person goes "oh ok, I understand that. I'll do it."
aaron695 3y ago
[dead]
brigandish 3y ago
Who is surprised that in a match between a doctor's bedside manner and a machine's that a machine wins? The cold metal of a machine is nothing next to the coldness of a doctor's heart! ;)
- JaneLovesDotNet 3y ago
  There is something comforting about how ChatGPT never becomes overly emotional. I'm not surprised it outperforms humans in bedside manner except for occasions where the human is particularly adept.
- kmoser 3y ago
  My takeaway isn't how useful ChatGPT is, but how useless the doctor is, and by extension, how poorly the the field of medicine is that it doesn't adequately train doctors to have better bedside manner. Regardless of my state of agitation, I would most definitely not want to be read a script prepared by an AI. In fact, were I to find out after the fact, I'd be even more incensed.
  - kaitai 3y ago
    I passed this article & discussion on to physician spouse, who would specifically like me to reply to this comment: "You are being very emotional." Digging deeper (as the comment above alone isn't worthy of HN, IMO): "People believe what they want to believe because of small sample sizes. When there isn't a black & white answer people go back to their personal experience, which is limited. They will not be right in the way you want to be right if you're practicing medicine." The family members are certainly exhibiting this. Are any of us, in this discussion? Additional question: "Who is the patient? Is it the anxious family members who are being treated, or the person with fluid in the lungs?" The person with fluid in the lungs is the patient and the doc needs to be spending time on that person (the one who will die if not treated correctly). The family members are important but not in need of medical care. They can be handed off to an AI in the short term. From my own vantage point, I've observed that the context switching between "caring for patient" and "dealing with family" can put a substantial drag on cognitive function for physicians. They do need to suck it up and deal with it since family members are the primary way in which patients receive post-release care in the US due to the non-existent safety net, and family member presence can really improve outcomes, but I'm not surprised that ChatGPT helps in producing the sort of bland prose that is well-received by family members.
deleted 3y ago
[deleted]
sarchertech 3y ago
There is no way it was faster to type this, verify it, and print it out than it was to just stop and think what you wanted to say. I also very much doubt an agitated family member will be calmed down by a doctor reading a script to them.
harshaw 3y ago
The point here is that he is using chatGPT /LLMs as an assistant - not making a diagnosis or something. He easily could have come up with a similar script from working with another member of hospital staff (maybe a social worker??) but that takes time. What I want (in a business context) is a LLM that can rewrite memos / docs (I am an Amazonian) in exactly the style that someone expects and is trained on other successful docs. It's not to cut out the hard work of getting the details right - its to cut down on the bullshit of misalignment from bad communication.
firebirdn99 3y ago
This really feels farcical, and kind of like vc bros hyping up a product. Considering this was a serious issue of life and death of a patient, it was surprising how it read like a prank or like the scene from Harold and Kumar, where Kumar plays a fake doctor and saves an ER patient through sheer luck. I know some doctors are full of baloney but c'mon. I really hope maybe that we create better doctors that have a lot of empathy, and are able to wield technology to make themselves 10x better like we're doing with programming. But, this is not the way. Edit: I realized its fake...I think, it has to be. It doesn't make sense, like others mention. You're not reading a 5 para speech to a patient's family while they are being treated. And also, in the article, they say to double check and verify what is being said. You better proofread that too! Or it might invent something absolutely crazy, and you may be liable.
- __loam 3y ago
  > are able to wield technology to make themselves 10x better like we're doing with programming lmfao
- seabombs 3y ago
  The author is a doctor, but also an executive at a health-focused VC fund, with several AI companies in it's portfolio. Helpful context to have before reading his article I think.
  - elliotpage 3y ago
    Gotta love a classic Conflict of Interest!
  - fifteenforty 3y ago
    The most unbelievable part of this story is that he claims to have read it out to the family.
    - zikzak 3y ago
      I am currently looking for a new Team Lead. I interviewed a guy yesterday who performed well in a soft skills interview and so this was his tech interview with a senior dev and myself. He was doing OK at first. As the questions got a little more nuanced, there were these pauses. Eventually, he got flustered and asked to resume later. We agreed to this since he said he'd been dealing with a production issue for several hours before the scheduled time (I had offered to reschedule at no detriment to him before we began but he insisted on doing it then). The developer and I met after this first interview and discussed his answers. We both thought he was "googling" (we are older people, stuck in the past). Later that day, back on another call with this guy, my senior dev starts typing my questions into ChatGPT and getting back essentially the answers he's giving us. I start with more nuanced questions and he's done. He just kept reiterating platitudes could not explain why he was saying them, etc. I felt bad for him, he was trying to get a job beyond his technical knowledge. But ultimately, this was dishonest and I would have found out very quickly he was not ready for the job. If anyone is wondering, I gave him a chance to tell me why he was pausing for so long before answering and why his answers didn't seem to directly address my questions and he just insisted he was NOT pausing (he was, for up to 30 seconds at a time). It was generally just a really disappointing experience for all of us, I would imagine.
mvdtnz 3y ago
If an ER doctor printed an AI-generated script and read it to me while I was in an extremely stressful situation I would punch him in the head. This is disgusting.
elforce002 3y ago
Experience, Knowledge, Accountability, and Human touch. No amount of "AI" is going to change that and I'm saying that as a software engineer.
- talldatethrow 3y ago
  As someone that knows several doctors very well (decades of friendship, weekly coffee or drinks), you are really over estimating the average patient doctor outside of basically surgeons. A doctor might have everything you describe. But they don't have the time, energy, or desire to spend much time discussing things with patients. Similar to a car mechanic. They know a lot. And if it's their car or their father's question, they can probably walk you through rebuilding an engine over the phone. But try getting a real answer out of one during a visit. The power of some of these AI bots is that you can query them to your hearts desire until you are really satisfied.
gerdesj 3y ago
"Understandably concerned, they hovered around my staff and me, to the point where their constant barrage of requests was actually slowing down treatment. To compound the problem, they were absolutely certain what their mother needed." This is diagnostics. Doctors do it and so do IT bods (and others) but our patients (computers) dying is a bit of a fiddle rather than a tragedy or catastrophe. I would not dream of letting CGPT loose on "indications/conta-indications" which is how medics work (IT could learn a lesson or two here instead of the usual "magic" thinking). CGPT has a learned data set and it is not perfect. It will spit out a series of words given an input. No one can really calculate the output given an input - its non deterministic. It does look like the output is quite intelligent. I consider CGPT as a really fancy calculator with all the usual caveats: do check the output carefully.
hartator 3y ago
It is kind of counterintuitive that IV fluids are bad for the lungs somehow. You can’t really blame the family. Specially when the “bro” doctor decides it’s time for ChatGPT.
Pigalowda 3y ago
This is silly and fake. Patient families are like this all the time. Just explain like a real big boy doctor
Tempest1981 3y ago
Is there a name for this 5 paragraph response format, that feels so intrinsic to chatGPT? (5 paragraphs: General empathy, specific, empathy, technical explanation, establishing trust, reassurance) Is the style a result of the default rules that OpenAI is using? Or is it derived automatically from the material chatGPT was trained on?
- TechBro8615 3y ago
  Yes, it's called the five-paragraph essay [0] :) Or at least, it's a variant of it, with more specific constraints on some of the paragraphs (e.g. "establishing empathy" in the introduction). [0] https://en.wikipedia.org/wiki/Five-paragraph_essay https://en.wikipedia.org/wiki/Five-paragraph_essay
boringuser2 3y ago
Well, I'm a dead man.
burnished 3y ago
I'm having a hard time buying this anecdote. Not even the bit where its implied that multiple people read the same script multiple times and that worked for soothing people, but the actual content of the before and after; the gpt version doesn't explain it any differently and the 'compassionate' writing was absurd. My most charitable interpretation is that the anecdote is true but the article itself was written by some one else and is more of an 'inspired by real events' situation. Side note; does anyone actually think that chat gpt succeeds at rewriting things 'compassionately' or 'more intelligently'? From what I have seen the actual output more resembles caricature than the requested results, in the same way that corporate communications frequently parody earnest communication.
- chocolatkey 3y ago
  > does anyone actually think that chat gpt succeeds at rewriting things 'compassionately' or 'more intelligently'? I've seen non-native English speakers (co-worker) use it to write lengthy apologies to customers, and I (native speaker) think it sounds over the top sincere and dramatic, especially when the same person tends to talk in a completely different manner. I believe this story, because the customer in this case was satisfied too
  - robbrown451 3y ago
    "and I (native speaker) think it sounds over the top sincere and dramatic, especially when the same person tends to talk in a completely different manner." Of course, you can actually tell ChatGPT to change its style and mannerisms. Here's an example when I was experimenting with this. It went a bit over the top in the other direction, but only because I was so explicit about asking to to be casual. https://chat.openai.com/share/07f6f9aa-de02-4a0c-8eeb-574eef5722a5 https://chat.openai.com/share/07f6f9aa-de02-4a0c-8eeb-574eef...
    - fastball 3y ago
      It's even pretty good at doing this multi-lingually. I live in Indonesia, and there is a sort of dialect used by various people from Jakarta that basically combines Indonesian and English in a way that is quite recognizeable when you hear it. The other day I asked ChatGPT to answer a question I had in Indonesian, and the answer was very formal (Google Translate generally has the same problem – the translations it gives you are way too formal for most speech). So I asked it to rephrase in this Jakartan slang it and did very well.
PlunderBunny 3y ago
Could the explanation be simply that, when the doctor came back with the piece of paper, the family thought it was from a textbook, or a second-opinion from another doctor that had been printed out? I.e. that it carried more weight because of that.
stereoradonc 3y ago
One of the worst write ups trying to drum up "support" for a product known to hallucinate. I couldn't imagine this could land up here!
bastardoperator 3y ago
TL;DR ChatGPT: "using empathy, tell the family why an iv is bad for someone with fluids in their lungs, keep it short and concise" "I understand your concerns. It's tough to see a loved one struggling. When there's fluid in the lungs, adding more fluid to the body via IV can make the problem worse. It's like adding more water to a cup that's already overflowing. The body needs to remove excess fluid, not add more. Your loved one's healthcare team is making decisions to help, not harm. Please keep asking questions – it's important to understand the process. We're all here with the same goal: to ensure the best possible care for your family member."
hfo 3y ago
Physician here with ER experience (and IT too). I think it's quite realistic, except the relatives being in the ER room, giving orders to nurses. But that might be different in other hospitals and countries. The reason for the edema is not given, perhaps cardiac insufficience, perhaps she didn’t take her pills (it is mentioned that she had dementia). Anyway, it’s possible that treatment has already been done but would take a few hours to fully set in, with the relatives impatiently waiting and seeing no improvement. Explaining medical things to patients and relatives is difficult and a subject on its own in medschool. In extraordinary circumstances and without being able to do much on their own, people filter things they are told to what they want to hear. Sometimes you get feedback on what the patient understood, it's surprising and also the potential base of a lawsuit. There is a reason that you have to document so many things in writing. So I think it’s a smart idea to use AI to generate simple-language messages that are easily understood. It doesn’t matter if the medical content of the text is 100% correct, the message here is: ”Calm down, stand back, I handle this”.
- rediguanayum 3y ago
  +1 Wife is an ER doc. She also agrees with the premise management that communicating medical facts to that patient and family is burdensome and time consuming, especially when there's pressure on the staff to see a high volume of patients. She has her own anecdotes similar to the story. There needs to be improvements in communication though she hasn't bought into ChatGPT being the right way to do this.
- lukko 3y ago
  I agree it's a realistic scenario. I always thought one of the most important things about being a junior doctor (FY1 / 2 in UK) was to be able to have these conversations with next of kin in a way that is empathic and easy for them to follow. If they are not processing the information then you have to allow for that, but I do think some relatives would react very badly if given a printout from an AI to explain a situation, patients always bring up if a doctor has googled their symptoms / rash in front of them, this is giving much more authority to the AI in a more sensitive and potentially difficult situation. I think the way AI is used in medicine has to be thought about very deeply, I don't think it should just be a fallback in time-pressured or difficult scenarios.
faangiq 3y ago
Absolute nonsense.
lovelearning 3y ago
> If you have any questions or concerns, don’t hesitate to reach out to the medical team. But isn't that what they were doing in the first place? More honest to suggest don't hesitate to reach out to ChatGPT.
camillomiller 3y ago
Another nice movie script written by Quentin Inventino
saargrin 3y ago
so he outsourced the one uniquiely human task of explaining stuff to concerned family to an AI?
gumballindie 3y ago
> using ChatGPT to help treat patients He’s not using it to treat anyone. He’s using it for generating text that then is (weirdly) read to patients. These ads for chatgpt are getting incredibly hilarious as of recently. But an improvement over the initial “chatgpt saved my dog’s life” type of nonsense.
ChaitanyaSai 3y ago
Human doctors using AI to sound more humane was not something I'd have guessed. Not just a one-off case https://www.nytimes.com/2023/06/12/health/doctors-chatgpt-artificial-intelligence.html https://www.nytimes.com/2023/06/12/health/doctors-chatgpt-ar... And it is fun getting chatgpt to inject compassion into unexpected scenarios. https://chat.openai.com/share/83a5b1f2-9b5a-4ebd-947c-b68fd26a1876 https://chat.openai.com/share/83a5b1f2-9b5a-4ebd-947c-b68fd2...
einpoklum 3y ago
tl;dr: I use it for religious effect. The masses view AI-generated text as magic and accept it as authoritative over what I tell them, so I have ChatGPT rephrase what I was going to tell them and have it read "What the AI said". They totally buy it and leave me alone.
deleted 3y ago
[deleted]
locallost 3y ago
I don't know if it's fake, but I would not bet on it. There is imho a large crisis of trust in all types of institutions, professions etc. Being a doctor has been demystified to an extent, for various reasons (e.g. the realization I've heard from some people that doctors didn't really know what was wrong and they just went from one guess to another, large amount of information being available giving people the wrong impression they know more than they do etc. etc.). It's also my personal explanation for the rise in alternative medicine, homeopathy, ivermectin treatments for covid, but also more broadly various forms of conspiracy theories. It's not far fetched that some people would trust "AI" more than a doctor who they don't trust or respect at all anymore. So fake or not, it definitely hits a nerve as something that is possible. On a tangent, I think this is actually quite interesting. If I was a sociologist, I would be interested in the root cause of this as much as a scientist or an engineer is interested in a bridge falling down. But I mostly see lamenting that the world has gone crazy.
al_be_back 3y ago
>> I printed this response up, and read it to the concerned family. As I recited ChatGPT’s words, their agitated expressions immediately melted into calm agreeability. not bad for a corny $0.5 hardback novel - but wtf was that about? If a Doctor read that script about my mother in ER, I'd be more insulated than had they printed the source code of a Linux Kernel module and proceeded to read it out in full, verbatim! So what one's a Doctor or another's a Lawyer and even involved in tech VC world? Artificial Intelligence is extremely complex and specialized field, plenty of learned professionals and VCs have been mesmerized then duped by "altruistic" tech projects in the filed of AI or Crypto. In the 80 and 90s automation tech in call centers was supposed to revolutionize customer support blah blah blah ... didn't work, people hate automation/robotic behavior, so the Industry switched to outsourcing calls to human operators in countries with cheaper salaries - it worked - and that drove other outsourcing innovations. I fear, chat bots and generative AI will fair the same - they're barking at the wrong tree.
- lukko 3y ago
  Yep, I think fundamentally it feels wrong. I think many families would see it as a sign of incompetence if the doctor cannot explain a diagnosis or treatment plan - time pressure is part of the job unfortunately, and people may not process difficult information in stressful situations with as much clarity, you have to allow for those things. Reading a script from ChatGPT is insulting. Patients hate when doctors just google their symptoms / rash in front of them, this is quite a step further. Printing out supporting information for families / patients is ok, but it needs to be validated – I think this is quite a dangerous use of generative AI for both the doctor and the patient.

275 points by SLHamlet 3y ago

Two shocking things about this article: The first: with his doctor hat on, he should understand that the value is the piece of paper and reading from it consistently to relatives who may have mild cognitive impairment. ChatGPT wrote essentially identical content to the words he'd been delivering himself. The professionally responsible thing to do would have been to type up or get someone to type up his _actual words_, print those out, and sign his name to them. He is -- for marketing hype cycle reasons -- projecting magic onto AI and ChatGPT when anyone who has ever known anyone close with mild cognitive impairment knows that the consistent, calmly repeated message is the magic. This is really irresponsible at worst, and empty, AI-hopey-changey nonsense at best. The second: everyone here suggesting that this is a valid or appropriate or even appealing use of AI. Look: if you can't do empathy (and not all of us can), hire an actual person who can do empathy! Make that their job, support them in it, celebrate their ability to do it and reward them properly. They are worthwhile people do who important jobs. Don't give the job to an AI to produce a fake veneer of empathy. If you can't understand why faking and automating empathy is not a substitute for actually trying to find real empathy in a role where someone could be an effective emotional bridge between two worlds, and if you can't see that this kind of bland/BS/automated sincerity application of AI is really toxic, I don't know what to say to you.

catchnear4321 3y ago
the flip side of this is that the model acts as a universal translator. not just for language but also for emotional valence and context. are you a doctor? worked in an er? there are a lot of strong words about what doctors should and shouldn’t do, with regard to empathy and compassion. there is a shocking lack of compassion or empathy in the very words demanding them. (there is also a rather large assumption that the doctor would individually be able to hire someone on a whim, to talk nice to people. in an er?) the value of the paper, of the words, is that it helped bridge a recognized communication divide. the purpose? a combination of compassion and treatment. help the well-meaning elderly children understand well enough to keep them from indirectly killing their mother.
- bemusedthrow75 3y ago
  > are you a doctor? worked in an er? No, I'm not. My dad -- who died with dementia -- was in medicine for his entire life. I am writing about what people with cognitive impairment tend to need from the perspective of reassuring, daily, a highly intelligent person who couldn't remember stuff. > there are a lot of strong words about what doctors should and shouldn’t do, with regard to empathy and compassion. there is a shocking lack of compassion or empathy in the very words demanding them. The bit about hiring someone with empathy was quite clearly in my second point, which was aimed at the HN audience, not at the doctor. (Though the guy who wrote the article isn't writing as a doctor in some medical publication, he is writing an executive at VC firm that backs AI startups, as other commenters have observed.) I have a lot of compassion for doctors. I don't have a lot of compassion or empathy for motivated reasoning in AI marketing bullshit written by well-remunerated people who are trying to force a technology into healthcare where it does not belong. And yes, the value of the piece of paper is that it bridged the divide. That's exactly what I said. It does it through consistency and constancy. ChatGPT has _no_ magical impact on that.
  - catchnear4321 3y ago
    > The bit about hiring someone with empathy was quite clearly in my second point, which was aimed at the HN audience, not at the doctor. then the assumption of ability to hire or impact hiring is broader. the article author is quite biased, yes. part of the problem with current “ai” - bias from all sides. chatgpt is not magical. it is a tool. a novel one. that many wish to sell. none of that lessens the possibilities it presents. nor the risks. but there are real, beneficial use cases. with a little time spent. not everyone wishes to put greed ahead of linear algebra, or math before compassion.
    - bemusedthrow75 3y ago
      > then the assumption of ability to hire or impact hiring is broader. It is, but so what? What I am suggesting is that using AI tools to fake empathy is worse than not being able to do empathy yourself. This should be evident. If you have a task that requires significant empathy (a patient advocate, a customer advocate, a serious complaints line, anything with extensive exposure to the emotions of your customer) and you feel unable to do that (not everyone can), the ethical, responsible thing to do is to hire someone who does that job properly. Professionalise caring. Don't just give the role to an AI that can fake it with a simulacrum of past human caring that it read on the (largely corporate) internet. We are heading down a dark path if we pretend that automated fake empathy is acceptable. Edit: and yes, people fake empathy all the time, they copy and paste it. Of course they do. But each time someone fakes empathy there is a glimmer of a chance that they will realise that it's a bad, inappropriate thing to do. There's a glimmer of a chance that they go back to their boss and say "you know, I feel bad brushing them off; what they are going through is real and awful and we should address this more personally". AI empathy will never do that. And we should feel very, very nervous about using AI to deliver highly bespoke fake empathy to eliminate any guilt from that.

Applejinx 3y ago

I'd bet money that the Medium article has also been, in part, rewritten by ChatGPT. Here's why: the story being told is that of a doctor, needing to communicate, being able to get through to 'regular folks' through talking down to them using ChatGPT (which is actually well suited to that task, once you acknowledge that hallucination can also mean losing stuff in translation). Then, the article itself, including the description of the loving family's reaction, is VERY characteristic of the way ChatGPT 'talks down'. It's hallucinatory happytalk that paints with a very broad brush, and delivers not the tone you want, but an abstracted 'popularized' tone likely to convince large numbers of stupid and gullible people. That tone's hard to miss, and it's popping out all over the article. Points for committing to the bit. However, what's pictured is clearly a level of Hell, or possibly an out-take from Idiocracy. You won't believe what happened next! :D

Xen9 3y ago
"...hallucinatory happytalk that paints with a very broad brush, and delivers not the tone you want, but an abstracted 'popularized' tone likely to convince large numbers of stupid and gullible people." You capture essence beautifully.
- Applejinx 3y ago
  Only because I've written many words and had very specific concepts, 'essence', to represent with words. What I'm saying is not 'popularized', it's a set of very specific points. You could GPT-ize what I said into 'feel-good talk' or 'hallmark cards' but you would lose the concept of 'the very broad brush does not deliver the tone you want, but a generalized one that will reach more people through being already based in how they see things'. If you need to get a mass audience to see a new thing, ChatGPT is NOT your friend… it will tell them what they already know… the wild thing is how the original claim is just this, 'ChatGPT got an audience to see a new thing', but the ChatGPT-style payoff of 'they smiled bovinely and accepted the new information' is clearly what the ARTICLE AUTHOR wanted to hear, and thus it's a really suspicious take.
- easyThrowaway 3y ago
  A quick way I've found to recognize ChatGPT-written text is upon reading it aloud, it 's kinda obvious how constant and monotone it sounds, no matter the chosen style. It gives a very "metronomic" rhythm, almost like a krautrock song. It's kinda obvious once you notice it.

isykt 3y ago

> But in these scenarios, reviewing existing work is usually much faster than starting from scratch. This is true, but it’s also a different skill than writing is. There are people who write well enough but are terrible editors. I fear for them in the generative AI workplace.

kwhitefoot 3y ago

Sounds to me as though the patient's children simply regarded the printed word as more authoritative.

stcroixx 3y ago

I just don’t believe he read that to their face and didn’t get punched. No way this happened.

scrum-treats 3y ago

I can't wait to go to the doctor, they ask me my favorite TV show/movie/video game, and then I get my diagnosis explained to me within the context of that show/movie/game. I could ask GPT to "doctor up" the explanation, and ChatGPT gives me a little more technical/medical jargon. Or just straight cascades of analogy.

scrum-treats 3y ago
This is sarcasm, by the way.

paulryanrogers 3y ago

> ...only if you feed it perfect information, and the actual patient has a classic presentation, which is rarely the case. (And you don’t mind a 50% success rate that often misses life-threatening conditions.) This sounds like a step backwards?

danmaw 3y ago
It isn't a doctor, it's trained on Reddit threads and other random shit off the internet not medical journals.
- jprafael 3y ago
  Given that "The Pile" includes PubMed and NIH as data sources it would be unlikely to have GPT4 not use them at all. Even GPT3 uses Wikipedia which does have (mostly) factual data with cited sources.
  - ekianjo 3y ago
    Problem is half of the publications or even more is pure garbage that cannot be reproduced so there is that.
  - tazjin 3y ago
    > Even GPT3 uses Wikipedia which does have (mostly) factual data with cited sources. There's a LOT of stuff on Wikipedia where the source is a link to some random, long article and it's unclear where exactly the referred to information is coming from. Gets significantly worse for any "hot" topic.
  - kaliqt 3y ago
    It's lossy though, it tries to remember everything and relate that to everything else. Performance would increase significantly if it were tuned for that use case.
- fnordpiglet 3y ago
  You should try asking it detailed questions about medicine. It’s fairly deep and broad in its medical “knowledge.” But it’s not good at deductive or inductive reasoning and has no agency, so is entirely unsurprising it’s not good at differential diagnosis. We actually have good differential diagnosis expert systems the challenge is getting the providers to input queries properly. I can imagine GPT4 acting as an intermediary between human natural language and expert systems to great effect. It seems rather rash to be judging these technologies based off a crappy web UI thrown on a chatbot a few months ago.

pedalpete 3y ago

Will there be a Dr House (is there a more current TV doctor) video version which the family can interact with which would make them feel like they are getting the best care in the world from a famous doctor? But also, if ChatGPT has a 50% chance of giving the wrong diagnosis, can you trust it more than 50% to give the correct reasoning to the family as to the activity of the medical staff?

weaksauce 3y ago
no, you still have to read it. > In each case, the output from the hungover intern/ChatGPT needs to be carefully checked before it’s used. But in these scenarios, reviewing existing work is usually much faster than starting from scratch.
tazjin 3y ago
> if ChatGPT has a 50% chance of giving the wrong diagnosis It depends on the average chance of the doctor giving the wrong diagnosies (I honestly have no idea?!)
- OJFord 3y ago
  My experience is second-hand, but they seem to be pretty split between seeing exciting rare diagnoses everywhere (students, juniors primarily, or one's own particular research interest) and biased against them, it's ~never that, it's the same boring thing we saw ten of yesterday (jaded specialists primarily).
- EA-3167 3y ago
  There are different figures, partly as a result of how this question is studied. The Mayo Clinic did a famous study on people who sought a second opinion, and they found that something like 12% of people seeking that second opinion were correctly diagnosed. Obviously this isn't a fair sample since this is not only people seeking a second opinion, but one from a prestigious source. This narrative review discusses the issue from a global perspective, concluding that most people will experience at least one misdiagnosis. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5502242/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5502242/ Article on Mayo study, with link to study: https://newsnetwork.mayoclinic.org/discussion/mayo-clinic-researchers-demonstrate-value-of-second-opinions/ https://newsnetwork.mayoclinic.org/discussion/mayo-clinic-re...

ilaksh 3y ago

Theoretically something like BioMedLM and/or a vector search over clinical research and/or GPT-4 with cited sources etc. could be useful in some diagnostic circumstances to at least give doctors ideas, right? What he is talking about here is just an incredibly common scenario that the doctor and everyone is already very familiar with. Which is basically delaying the inevitable for people who are in some ways already gone. Also my understanding is that the ChatGPT API does have some level of privacy now although maybe not compliant with the privacy laws which seems somewhat burdensome on the entire medical industry (overcompensating in some detrimental ways). It seems like the fundamental problem is aging. Maybe a leading edge LLM approach can help keep people up to speed on aging research. Although probably that's pushing it because you would need to get very deep into mitochondrial damage or whatever.

wolverine876 3y ago
> Theoretically something like BioMedLM and/or a vector search over clinical research and/or GPT-4 with cited sources etc. could be useful in some diagnostic circumstances to at least give doctors ideas, right? Anything is possible, but consider this doctor's review of ChatGPT's diagonistic output: > ChatGPT works fairly well as a diagnostic assistant — but only if you feed it perfect information, and the actual patient has a classic presentation, which is rarely the case. (And you don’t mind a 50% success rate that often misses life-threatening conditions.) For any new technology to be useful, it needs to do more than just work; it must be significantly more productive than the alternative.
- ilaksh 3y ago
  That review is one anecdote which does not specify the ChatGPT model used (which vary widely in performance), is not BioMedLM, is not a vector search, etc.

Animats 3y ago

When you need blithering, ChatGPT is there for you.

wolverine876 3y ago
You are just in wishful denial about the technological disruption: ChatGPT will rapidly and dramatically increase productivity by displacing human workers. ChatGPT will automate 99.9% of what's on the Internet, which is blithering, displacing what most human labor is spent on these days, and leaving humans with no demand for their labor but going back to producing accurate, valuable knowledge. The halcyon days are over. Post-truth technology has followed the typical maturation cycle, and now is at the point that we can automate it. (HN is dead. This post could be written by ChatGPT or a future LLM.) ;) :DDDDDDD
TeMPOraL 3y ago
In other words: ChatGPT is there for you to help with most human interaction.

sfink 3y ago

Kind of fascinating that the benefit here is not so much artificial intelligence as artificial empathy.

VWWHFSfQ 3y ago
"here's a squeeze ball. go sit outside and squeeze it."
tshadley 3y ago
All cynicism aside, there's vastly more in the collective writings of humans on empathy than medicine.
- version_five 3y ago
  See my sibling comment just posted, I don't belive that's relevant. You can always screw up facts, it's a high bar. A loosely constrained example of an empathetic speech much easier. Despite the fact that medicine deals in facts (and this would hold true for an even more black and white discipline) so all you need in principle to learn from are a few examples, while empathy is much more subtle and would require a representation to be learned from many varied examples.
  - fsckboy 3y ago
    > the fact that medicine deals in facts medicine benefits from facts, but is largely based on experienced, expert opinion
    - riwsky 3y ago
      And experience is the practical contact with and observation of facts, over an extended period. Most English dictionaries explicitly separate out “formal judgment by an expert” as a distinct meaning for “opinion” from the more common meaning of personal subjective no-facts-needed belief.
Nition 3y ago

amstan 3y ago

> Satisfying HIPAA rules around patient privacy alone may take many years, or decades, to resolve themselves before we could even contemplate directly using programs like ChatGPT in a medical theater. This is easily solved by not using someone else's API. Here's $500 worth of hardware you can get on ebay running Vicuna-13B, locally: > IV fluids may not be the best treatment for someone with severe pulmonary edema and respiratory distress because it could make their condition worse. This is because when a person has severe pulmonary edema, their lungs are already filled with fluid, which makes it harder for them to breathe. Adding more fluid to their body could increase the pressure in their lungs and make it even harder for them to breathe. This could be life-threatening. > Instead, the best course of action would be to focus on treating the underlying cause of the pulmonary edema, such as heart failure or a lung infection. This may involve giving medications to help the heart pump more effectively or to reduce inflammation in the lungs. The patient may also need oxygen therapy to help them breathe more easily. > It's important to remember that every case is unique, and the best course of treatment will depend on the individual patient's condition. The healthcare team will do their best to provide the most appropriate care for the patient based on their specific needs.

cced 3y ago
How much time does this take to get produced locally?
- smoldesu 3y ago
  I run a similar Vicuna bot on a free ARM VPS from Oracle. Inference takes ~20 seconds for ~200 tokens on the CPU, so it should stream results about half as fast as ChatGPT. ...however, that was on cheapo ARM hardware. On a hospital budget I bet you could beat OpenAI with a local model pre-mapped in GPU memory on a 3090.
  - cced 3y ago
    Do you guys have any reference material on how to get this started?

deleted 3y ago

[deleted]

deleted 3y ago
[deleted]

CodeWriter23 3y ago

I’d say doc should be expecting a visit from the FDA shortly about his unlicensed medical device.

ftxbro 3y ago
maybe the FDA is busy visiting every doc that uses google
- CodeWriter23 3y ago
  Are you suggesting ChatGPT is merely an overhyped search engine?
  - ftxbro 3y ago
    no, you said that. i'm saying that if gpt is a medical device then so is google, and doctors use google all the time
captainoats 3y ago
The FDA regulates bringing a device to market commercially, doctors can use regular COTS software.

peter_l_downs 3y ago

This is the most “that didn’t happen” story I’ve seen in a while. No way the concerned family responded positively to the doctor reading a script to them off a clipboard, forgot about it, and then responded positively to other nurses/doctors reading the same script to them again.

bagels 3y ago
My wife works in cardiac and sees similarly confused patients and elderly family members. Have you tried to have conversations with people experiencing dementia?
- peter_l_downs 3y ago
  Yes, it is unfortunate and difficult and I did not realize reading a prepared script off a clipboard would have helped me get my point across better. I could only see it causing them to view me as stonewalling them or cause them further confusion. I may have to try this for myself though, I am not a doctor and I haven’t tried it before.
- epcoa 3y ago
  Yes, many times. Certainly don't do it by reading off a preprinted script.
talldatethrow 3y ago
You've clearly never worked in sales. Sometimes you simply can't make progress and another person comes in saying the exact same thing and the person goes "oh ok, I understand that. I'll do it."

aaron695 3y ago

[dead]

brigandish 3y ago

Who is surprised that in a match between a doctor's bedside manner and a machine's that a machine wins? The cold metal of a machine is nothing next to the coldness of a doctor's heart! ;)

JaneLovesDotNet 3y ago
There is something comforting about how ChatGPT never becomes overly emotional. I'm not surprised it outperforms humans in bedside manner except for occasions where the human is particularly adept.
kmoser 3y ago
My takeaway isn't how useful ChatGPT is, but how useless the doctor is, and by extension, how poorly the the field of medicine is that it doesn't adequately train doctors to have better bedside manner. Regardless of my state of agitation, I would most definitely not want to be read a script prepared by an AI. In fact, were I to find out after the fact, I'd be even more incensed.
- kaitai 3y ago
  I passed this article & discussion on to physician spouse, who would specifically like me to reply to this comment: "You are being very emotional." Digging deeper (as the comment above alone isn't worthy of HN, IMO): "People believe what they want to believe because of small sample sizes. When there isn't a black & white answer people go back to their personal experience, which is limited. They will not be right in the way you want to be right if you're practicing medicine." The family members are certainly exhibiting this. Are any of us, in this discussion? Additional question: "Who is the patient? Is it the anxious family members who are being treated, or the person with fluid in the lungs?" The person with fluid in the lungs is the patient and the doc needs to be spending time on that person (the one who will die if not treated correctly). The family members are important but not in need of medical care. They can be handed off to an AI in the short term. From my own vantage point, I've observed that the context switching between "caring for patient" and "dealing with family" can put a substantial drag on cognitive function for physicians. They do need to suck it up and deal with it since family members are the primary way in which patients receive post-release care in the US due to the non-existent safety net, and family member presence can really improve outcomes, but I'm not surprised that ChatGPT helps in producing the sort of bland prose that is well-received by family members.

deleted 3y ago

[deleted]

sarchertech 3y ago

There is no way it was faster to type this, verify it, and print it out than it was to just stop and think what you wanted to say. I also very much doubt an agitated family member will be calmed down by a doctor reading a script to them.

harshaw 3y ago

The point here is that he is using chatGPT /LLMs as an assistant - not making a diagnosis or something. He easily could have come up with a similar script from working with another member of hospital staff (maybe a social worker??) but that takes time. What I want (in a business context) is a LLM that can rewrite memos / docs (I am an Amazonian) in exactly the style that someone expects and is trained on other successful docs. It's not to cut out the hard work of getting the details right - its to cut down on the bullshit of misalignment from bad communication.

firebirdn99 3y ago

This really feels farcical, and kind of like vc bros hyping up a product. Considering this was a serious issue of life and death of a patient, it was surprising how it read like a prank or like the scene from Harold and Kumar, where Kumar plays a fake doctor and saves an ER patient through sheer luck. I know some doctors are full of baloney but c'mon. I really hope maybe that we create better doctors that have a lot of empathy, and are able to wield technology to make themselves 10x better like we're doing with programming. But, this is not the way. Edit: I realized its fake...I think, it has to be. It doesn't make sense, like others mention. You're not reading a 5 para speech to a patient's family while they are being treated. And also, in the article, they say to double check and verify what is being said. You better proofread that too! Or it might invent something absolutely crazy, and you may be liable.

__loam 3y ago
> are able to wield technology to make themselves 10x better like we're doing with programming lmfao
seabombs 3y ago
The author is a doctor, but also an executive at a health-focused VC fund, with several AI companies in it's portfolio. Helpful context to have before reading his article I think.
- elliotpage 3y ago
  Gotta love a classic Conflict of Interest!
- fifteenforty 3y ago
  The most unbelievable part of this story is that he claims to have read it out to the family.
  - zikzak 3y ago
    I am currently looking for a new Team Lead. I interviewed a guy yesterday who performed well in a soft skills interview and so this was his tech interview with a senior dev and myself. He was doing OK at first. As the questions got a little more nuanced, there were these pauses. Eventually, he got flustered and asked to resume later. We agreed to this since he said he'd been dealing with a production issue for several hours before the scheduled time (I had offered to reschedule at no detriment to him before we began but he insisted on doing it then). The developer and I met after this first interview and discussed his answers. We both thought he was "googling" (we are older people, stuck in the past). Later that day, back on another call with this guy, my senior dev starts typing my questions into ChatGPT and getting back essentially the answers he's giving us. I start with more nuanced questions and he's done. He just kept reiterating platitudes could not explain why he was saying them, etc. I felt bad for him, he was trying to get a job beyond his technical knowledge. But ultimately, this was dishonest and I would have found out very quickly he was not ready for the job. If anyone is wondering, I gave him a chance to tell me why he was pausing for so long before answering and why his answers didn't seem to directly address my questions and he just insisted he was NOT pausing (he was, for up to 30 seconds at a time). It was generally just a really disappointing experience for all of us, I would imagine.

mvdtnz 3y ago

If an ER doctor printed an AI-generated script and read it to me while I was in an extremely stressful situation I would punch him in the head. This is disgusting.

elforce002 3y ago

Experience, Knowledge, Accountability, and Human touch. No amount of "AI" is going to change that and I'm saying that as a software engineer.

talldatethrow 3y ago
As someone that knows several doctors very well (decades of friendship, weekly coffee or drinks), you are really over estimating the average patient doctor outside of basically surgeons. A doctor might have everything you describe. But they don't have the time, energy, or desire to spend much time discussing things with patients. Similar to a car mechanic. They know a lot. And if it's their car or their father's question, they can probably walk you through rebuilding an engine over the phone. But try getting a real answer out of one during a visit. The power of some of these AI bots is that you can query them to your hearts desire until you are really satisfied.

gerdesj 3y ago

"Understandably concerned, they hovered around my staff and me, to the point where their constant barrage of requests was actually slowing down treatment. To compound the problem, they were absolutely certain what their mother needed." This is diagnostics. Doctors do it and so do IT bods (and others) but our patients (computers) dying is a bit of a fiddle rather than a tragedy or catastrophe. I would not dream of letting CGPT loose on "indications/conta-indications" which is how medics work (IT could learn a lesson or two here instead of the usual "magic" thinking). CGPT has a learned data set and it is not perfect. It will spit out a series of words given an input. No one can really calculate the output given an input - its non deterministic. It does look like the output is quite intelligent. I consider CGPT as a really fancy calculator with all the usual caveats: do check the output carefully.

hartator 3y ago

It is kind of counterintuitive that IV fluids are bad for the lungs somehow. You can’t really blame the family. Specially when the “bro” doctor decides it’s time for ChatGPT.

Pigalowda 3y ago

This is silly and fake. Patient families are like this all the time. Just explain like a real big boy doctor

Tempest1981 3y ago

Is there a name for this 5 paragraph response format, that feels so intrinsic to chatGPT? (5 paragraphs: General empathy, specific, empathy, technical explanation, establishing trust, reassurance) Is the style a result of the default rules that OpenAI is using? Or is it derived automatically from the material chatGPT was trained on?

TechBro8615 3y ago
Yes, it's called the five-paragraph essay [0] :) Or at least, it's a variant of it, with more specific constraints on some of the paragraphs (e.g. "establishing empathy" in the introduction). [0] https://en.wikipedia.org/wiki/Five-paragraph_essay https://en.wikipedia.org/wiki/Five-paragraph_essay

boringuser2 3y ago

Well, I'm a dead man.

burnished 3y ago

I'm having a hard time buying this anecdote. Not even the bit where its implied that multiple people read the same script multiple times and that worked for soothing people, but the actual content of the before and after; the gpt version doesn't explain it any differently and the 'compassionate' writing was absurd. My most charitable interpretation is that the anecdote is true but the article itself was written by some one else and is more of an 'inspired by real events' situation. Side note; does anyone actually think that chat gpt succeeds at rewriting things 'compassionately' or 'more intelligently'? From what I have seen the actual output more resembles caricature than the requested results, in the same way that corporate communications frequently parody earnest communication.

chocolatkey 3y ago
> does anyone actually think that chat gpt succeeds at rewriting things 'compassionately' or 'more intelligently'? I've seen non-native English speakers (co-worker) use it to write lengthy apologies to customers, and I (native speaker) think it sounds over the top sincere and dramatic, especially when the same person tends to talk in a completely different manner. I believe this story, because the customer in this case was satisfied too
- robbrown451 3y ago
  "and I (native speaker) think it sounds over the top sincere and dramatic, especially when the same person tends to talk in a completely different manner." Of course, you can actually tell ChatGPT to change its style and mannerisms. Here's an example when I was experimenting with this. It went a bit over the top in the other direction, but only because I was so explicit about asking to to be casual. https://chat.openai.com/share/07f6f9aa-de02-4a0c-8eeb-574eef5722a5 https://chat.openai.com/share/07f6f9aa-de02-4a0c-8eeb-574eef...
  - fastball 3y ago
    It's even pretty good at doing this multi-lingually. I live in Indonesia, and there is a sort of dialect used by various people from Jakarta that basically combines Indonesian and English in a way that is quite recognizeable when you hear it. The other day I asked ChatGPT to answer a question I had in Indonesian, and the answer was very formal (Google Translate generally has the same problem – the translations it gives you are way too formal for most speech). So I asked it to rephrase in this Jakartan slang it and did very well.

PlunderBunny 3y ago

Could the explanation be simply that, when the doctor came back with the piece of paper, the family thought it was from a textbook, or a second-opinion from another doctor that had been printed out? I.e. that it carried more weight because of that.

stereoradonc 3y ago

One of the worst write ups trying to drum up "support" for a product known to hallucinate. I couldn't imagine this could land up here!

bastardoperator 3y ago

TL;DR ChatGPT: "using empathy, tell the family why an iv is bad for someone with fluids in their lungs, keep it short and concise" "I understand your concerns. It's tough to see a loved one struggling. When there's fluid in the lungs, adding more fluid to the body via IV can make the problem worse. It's like adding more water to a cup that's already overflowing. The body needs to remove excess fluid, not add more. Your loved one's healthcare team is making decisions to help, not harm. Please keep asking questions – it's important to understand the process. We're all here with the same goal: to ensure the best possible care for your family member."

hfo 3y ago

Physician here with ER experience (and IT too). I think it's quite realistic, except the relatives being in the ER room, giving orders to nurses. But that might be different in other hospitals and countries. The reason for the edema is not given, perhaps cardiac insufficience, perhaps she didn’t take her pills (it is mentioned that she had dementia). Anyway, it’s possible that treatment has already been done but would take a few hours to fully set in, with the relatives impatiently waiting and seeing no improvement. Explaining medical things to patients and relatives is difficult and a subject on its own in medschool. In extraordinary circumstances and without being able to do much on their own, people filter things they are told to what they want to hear. Sometimes you get feedback on what the patient understood, it's surprising and also the potential base of a lawsuit. There is a reason that you have to document so many things in writing. So I think it’s a smart idea to use AI to generate simple-language messages that are easily understood. It doesn’t matter if the medical content of the text is 100% correct, the message here is: ”Calm down, stand back, I handle this”.

rediguanayum 3y ago
+1 Wife is an ER doc. She also agrees with the premise management that communicating medical facts to that patient and family is burdensome and time consuming, especially when there's pressure on the staff to see a high volume of patients. She has her own anecdotes similar to the story. There needs to be improvements in communication though she hasn't bought into ChatGPT being the right way to do this.
lukko 3y ago
I agree it's a realistic scenario. I always thought one of the most important things about being a junior doctor (FY1 / 2 in UK) was to be able to have these conversations with next of kin in a way that is empathic and easy for them to follow. If they are not processing the information then you have to allow for that, but I do think some relatives would react very badly if given a printout from an AI to explain a situation, patients always bring up if a doctor has googled their symptoms / rash in front of them, this is giving much more authority to the AI in a more sensitive and potentially difficult situation. I think the way AI is used in medicine has to be thought about very deeply, I don't think it should just be a fallback in time-pressured or difficult scenarios.

faangiq 3y ago

Absolute nonsense.

lovelearning 3y ago

> If you have any questions or concerns, don’t hesitate to reach out to the medical team. But isn't that what they were doing in the first place? More honest to suggest don't hesitate to reach out to ChatGPT.

camillomiller 3y ago

Another nice movie script written by Quentin Inventino

saargrin 3y ago

so he outsourced the one uniquiely human task of explaining stuff to concerned family to an AI?

gumballindie 3y ago

> using ChatGPT to help treat patients He’s not using it to treat anyone. He’s using it for generating text that then is (weirdly) read to patients. These ads for chatgpt are getting incredibly hilarious as of recently. But an improvement over the initial “chatgpt saved my dog’s life” type of nonsense.

ChaitanyaSai 3y ago

Human doctors using AI to sound more humane was not something I'd have guessed. Not just a one-off case https://www.nytimes.com/2023/06/12/health/doctors-chatgpt-artificial-intelligence.html https://www.nytimes.com/2023/06/12/health/doctors-chatgpt-ar... And it is fun getting chatgpt to inject compassion into unexpected scenarios. https://chat.openai.com/share/83a5b1f2-9b5a-4ebd-947c-b68fd26a1876 https://chat.openai.com/share/83a5b1f2-9b5a-4ebd-947c-b68fd2...

einpoklum 3y ago

tl;dr: I use it for religious effect. The masses view AI-generated text as magic and accept it as authoritative over what I tell them, so I have ChatGPT rephrase what I was going to tell them and have it read "What the AI said". They totally buy it and leave me alone.

deleted 3y ago

[deleted]

locallost 3y ago

I don't know if it's fake, but I would not bet on it. There is imho a large crisis of trust in all types of institutions, professions etc. Being a doctor has been demystified to an extent, for various reasons (e.g. the realization I've heard from some people that doctors didn't really know what was wrong and they just went from one guess to another, large amount of information being available giving people the wrong impression they know more than they do etc. etc.). It's also my personal explanation for the rise in alternative medicine, homeopathy, ivermectin treatments for covid, but also more broadly various forms of conspiracy theories. It's not far fetched that some people would trust "AI" more than a doctor who they don't trust or respect at all anymore. So fake or not, it definitely hits a nerve as something that is possible. On a tangent, I think this is actually quite interesting. If I was a sociologist, I would be interested in the root cause of this as much as a scientist or an engineer is interested in a bridge falling down. But I mostly see lamenting that the world has gone crazy.

al_be_back 3y ago

>> I printed this response up, and read it to the concerned family. As I recited ChatGPT’s words, their agitated expressions immediately melted into calm agreeability. not bad for a corny $0.5 hardback novel - but wtf was that about? If a Doctor read that script about my mother in ER, I'd be more insulated than had they printed the source code of a Linux Kernel module and proceeded to read it out in full, verbatim! So what one's a Doctor or another's a Lawyer and even involved in tech VC world? Artificial Intelligence is extremely complex and specialized field, plenty of learned professionals and VCs have been mesmerized then duped by "altruistic" tech projects in the filed of AI or Crypto. In the 80 and 90s automation tech in call centers was supposed to revolutionize customer support blah blah blah ... didn't work, people hate automation/robotic behavior, so the Industry switched to outsourcing calls to human operators in countries with cheaper salaries - it worked - and that drove other outsourcing innovations. I fear, chat bots and generative AI will fair the same - they're barking at the wrong tree.

lukko 3y ago
Yep, I think fundamentally it feels wrong. I think many families would see it as a sign of incompetence if the doctor cannot explain a diagnosis or treatment plan - time pressure is part of the job unfortunately, and people may not process difficult information in stressful situations with as much clarity, you have to allow for those things. Reading a script from ChatGPT is insulting. Patients hate when doctors just google their symptoms / rash in front of them, this is quite a step further. Printing out supporting information for families / patients is ok, but it needs to be validated – I think this is quite a dangerous use of generative AI for both the doctor and the patient.