6 ms·

Given my experience with LLMs I am skeptical that enough prompt engineering and tuning can adequately summarize patient records. I don't expect an LLM to adequa

Given my experience with LLMs I am skeptical that enough prompt engineering and tuning can adequately summarize patient records. I don't expect an LLM to adequately interpret data in a way that is useful to me. While much is missed on skimming records, an AI summary sounds like a false sense of security.

mountainriver 2y ago
Yup I actually worked on this problem and we abandoned it because the downside of missing important patient information was too risky
mousetree 2y ago
I recently received a 2 page written report of an MRI. I couldn’t understand a single word of it. While waiting a few weeks for my appointment at the next specialist I used ChatGPT to explain it and I found it to be very helpful in understanding what was going on. I felt it did a better job in explaining in lay terms than the specialist doctors did (radiologists, orthopedic surgeons, rheumatologists).
- SaberTail 2y ago
  How do you know ChatGPT explained it to you correctly? How do you know it didn't give you a wrong explanation that sounds plausible to a lay person?
  - renewiltord 2y ago
    For my part I know the same because I did it and then gave both the content and the generated report to my father: an orthopaedic trauma surgeon / my cousin a radiologist and surgeon / my other cousin a cardiovascular surgeon. All for different things. Varying degrees of enthusiasm (my father was the most thrilled) but all remarked it was accurate. Of course I did these things for a laugh because I can just access the appropriate professional on-demand.
  - CharlieDigital 2y ago
    There are at least two distinct workloads for ChatGPT: 1) No context, general knowledge workloads, 2) Contextual workloads using either RAG or direct input of context. In general, GPT will not hallucinate or give bad responses on the latter since it's working from a specific corpus of information (whether from RAG or from some context directly provided by the user). This is, of course, not foolproof since some of it is dependent on a good prompt and some of it is dependent on the question (GPT is well known to be poor at math, for example). But for summarization, it is exceedingly good. The former is where it can tend to hallucinate and generate bad responses.
danenania 2y ago
Something to keep in mind with every AI startup is that they’re trying to skate to where the puck will be in a year or two. You could still be right, but it’s worth considering what happens to any particular use case if the model gets 2 or 3x smarter.
- Zenzero 2y ago
  The difficulty partially lies in that there are numerous different "flavors" of medicine in any given specialty. Coupled with the vast amount of medical knowledge that is known in part or simply hypothesized given current evidence, you are going to run up against doctors disagreeing on how the summary selects what information can be typically left out. Even for myself, there is individual variation case to case what I would want to see. Sometimes it depends on things that aren't measurable. If my brain is exhausted after a long week, or if I couldn't sleep on Wednesday, I may conduct rounds a bit differently than if I am fresh. An LLM that isn't responsive in the same way can be problematic.
hehdhdjehehegwv 2y ago
I’ve likewise investigated this space, or an adjacent problem. TLDR: you can get good results with very high probability. However, medicine and law are not areas where “probably” even at a 95% level is going to cut it. You need 100%. (Sure a person makes mistakes as well, but we understand that and have systems in place to catch and correct. Whereas people are just tossing out LLM chatbots that are basically black box). It’s very similar to self driving cars, getting to 95% isn’t too hard now, but that last 5% to not accidentally kill a few people here and there is WAY WAY WAY harder.
Suppafly 2y ago
>Given my experience with LLMs I am skeptical that enough prompt engineering and tuning can adequately summarize patient records. What sort of experience do you have though? I'd say anyone familiar with the use of AI specifically as it related to medical records would probably disagree with you.
- Zenzero 2y ago
  I'm both a doctor and a SWE.

mountainriver 2y ago

Yup I actually worked on this problem and we abandoned it because the downside of missing important patient information was too risky

mousetree 2y ago

I recently received a 2 page written report of an MRI. I couldn’t understand a single word of it. While waiting a few weeks for my appointment at the next specialist I used ChatGPT to explain it and I found it to be very helpful in understanding what was going on. I felt it did a better job in explaining in lay terms than the specialist doctors did (radiologists, orthopedic surgeons, rheumatologists).

SaberTail 2y ago
How do you know ChatGPT explained it to you correctly? How do you know it didn't give you a wrong explanation that sounds plausible to a lay person?
- renewiltord 2y ago
  For my part I know the same because I did it and then gave both the content and the generated report to my father: an orthopaedic trauma surgeon / my cousin a radiologist and surgeon / my other cousin a cardiovascular surgeon. All for different things. Varying degrees of enthusiasm (my father was the most thrilled) but all remarked it was accurate. Of course I did these things for a laugh because I can just access the appropriate professional on-demand.
- CharlieDigital 2y ago
  There are at least two distinct workloads for ChatGPT: 1) No context, general knowledge workloads, 2) Contextual workloads using either RAG or direct input of context. In general, GPT will not hallucinate or give bad responses on the latter since it's working from a specific corpus of information (whether from RAG or from some context directly provided by the user). This is, of course, not foolproof since some of it is dependent on a good prompt and some of it is dependent on the question (GPT is well known to be poor at math, for example). But for summarization, it is exceedingly good. The former is where it can tend to hallucinate and generate bad responses.

danenania 2y ago

Something to keep in mind with every AI startup is that they’re trying to skate to where the puck will be in a year or two. You could still be right, but it’s worth considering what happens to any particular use case if the model gets 2 or 3x smarter.

Zenzero 2y ago
The difficulty partially lies in that there are numerous different "flavors" of medicine in any given specialty. Coupled with the vast amount of medical knowledge that is known in part or simply hypothesized given current evidence, you are going to run up against doctors disagreeing on how the summary selects what information can be typically left out. Even for myself, there is individual variation case to case what I would want to see. Sometimes it depends on things that aren't measurable. If my brain is exhausted after a long week, or if I couldn't sleep on Wednesday, I may conduct rounds a bit differently than if I am fresh. An LLM that isn't responsive in the same way can be problematic.

hehdhdjehehegwv 2y ago

I’ve likewise investigated this space, or an adjacent problem. TLDR: you can get good results with very high probability. However, medicine and law are not areas where “probably” even at a 95% level is going to cut it. You need 100%. (Sure a person makes mistakes as well, but we understand that and have systems in place to catch and correct. Whereas people are just tossing out LLM chatbots that are basically black box). It’s very similar to self driving cars, getting to 95% isn’t too hard now, but that last 5% to not accidentally kill a few people here and there is WAY WAY WAY harder.

Suppafly 2y ago

>Given my experience with LLMs I am skeptical that enough prompt engineering and tuning can adequately summarize patient records. What sort of experience do you have though? I'd say anyone familiar with the use of AI specifically as it related to medical records would probably disagree with you.

Zenzero 2y ago
I'm both a doctor and a SWE.