8 ms·

Clinical pharmacist for 10 years here. Yea, base model is very good. Better than first year residents - but not necessarily experienced clinicians. Now - throw

Clinical pharmacist for 10 years here. Yea, base model is very good. Better than first year residents - but not necessarily experienced clinicians. Now - throw a punch of clinical guidelines in a vector database and give it context and it’s 10x better than me and any doctor outside their speciality or all the mid-levels. (I.E, it’s better than cardiologist doing infectious disease - but not cardiologists doing cardiology). This because there are very niche stuff as you specialize where it’s only like 5 doctors who see it in the whole world on a consistent basis (and they don’t blog!) I trained it on the IDSA guidelines (infectious disease) and put up a proof of concept on GalenAI.co - just as way to start talking to health systems and clinicians. it’s going to be very different world in medicine in a couple of years from now!!

joshgel 4y ago
Ya, internist here. For some context, the USMLE is taken during medical school. The amount I have learned about actually practicing medicine since graduating is probably an order of magnitude more than everything I learned in medical school! I still learn stuff, all the time, and I’m not just talking about new research. So, while impressive and clearly part of the future world, we shouldn’t get too far ahead of ourselves with the current models. Edit: oh I should add that there are more clinically relevant exams that would be more likely to reveal d clinical usefulness, for example “board” exams. These are taken after training, usually before practice. Not knocking LLMs, just ensuring that people don’t misunderstand passing the USMLE as being clinically useful.
- capableweb 4y ago
  I agree that we shouldn't get ahead of ourselves with the current technology, but what you said earlier applies to practically every industry and science. What you learn at the actual job is always far more up to date than what you learn in school, no matter if it's being a engineer, doctor or just a lowly programmer.
  - notch898a 4y ago
    Yes but the difference is most engineers, and pretty much all lowly programmers are unlicensed. AI or some non-human accessible in the countryside so you don't have to order questionable "fish" antibiotics or "cat" anti-parasitic would be a nice step up from the current gatekeeping of medicine from people with limited access.
    - wbl 4y ago
      There are much simpler ways to provide access to those areas.
      - notch898a 4y ago
        The only significant ways I'm aware of that people get needed rx medications outside of physicians and mid-level practitioners is to leave the country or use "vet"/"animal" drugs. What are the simpler other alternatives currently available?
        wbl 4y ago
        Make more physicians and pay them to work in rural areas. Congress can just do this.
        goldenkey 4y ago
        American Medical Association is basically a union that artificially caps the number of doctors of each speciality.
        notch898a 4y ago
        The medical cartel loves to cloak their policies under the auspices of safety. In the end you'll find their policies magically result in massive profits for bureaucrats and chokepoints that constrain the supply of gatekeepers. This is not accidental.
- Madmallard 4y ago
  Ive never experienced anything in my care with doctors that I couldn’t understand with a days worth of research into things like UpToDate. It isn’t complicated. It is largely memorization and application of an algorithm which is just borderline useless for complex conditions that are emerging more modernly.
  - cguess 4y ago
    Have you ever had anything besides a bad cold? Tell me you understand every acronym in this article, and the ability to explain it succinctly to a patient much less to be able to hand off a case to "another" physician https://www.nejm.org/doi/full/10.1056/NEJMoa2206714 https://www.nejm.org/doi/full/10.1056/NEJMoa2206714 Doctors and nurses have saved me many times from some very close calls because of decades of experience, training and intuition. That is of course not to mention the friend who beat a deep brain tumor on their brain stem that everyone else told them was inoperable, and now are in medical school themselves for neurology. No LLM is going to pull that out of itself, possibly ever, and certainly not GPT-4 (no one else had ever had the surgery done before, it was novel).
    - Madmallard 4y ago
      I have an extremely large amount of experience in this context.
  - deleted 4y ago
    [deleted]
  - Veen 4y ago
    How do you know that you actually understand a topic as well as a doctor? How do you verify that? It's not unusual for people to think they comprehend a topic at expert level, when in fact they do not. The correlation between confidence and understanding is not a reliable measure. That's why doctors are trained by more expert colleagues who can judge their true understanding, have to take exams, etc.
    - Madmallard 4y ago
- sizzle 4y ago
  What if the AI is trained on board exams and other high signal testing/examination materials? Surely it will become superhuman in its medical abilities?
TaylorAlexander 4y ago
This makes me think we need some kind of program for experts to start writing things down in a way which is helpful. Even just take dictation and transcribe it.
- another_story 4y ago
  There are many tools for doctors to do just this, but it's a matter of time more than anything.
- andrewthornton 4y ago
  You should take a quick look at EPIC. They dominate the electronic heath record space, and a ton of health systems use it. You will know if your doctor's office uses an EHR application, because they will be typing notes into it for the majority of your visit. I have not been too excited about the amount of time that physicians spend on EHR systems, but I am hopeful that taking the data they input (along with blood work and other test results) will make everything more accurate, fast and effective.
  - jonathan-adly 4y ago
    EPIC unfortunately is all the bad things about Google, and none of the good. Unable to ship anything, protect their margin > help the users solve problems, monopoly, locked up distribution so no one else can innovate. Honestly, my bear case for AI in medicine is Epic picking up the phone and telling health-systems not to buy anything because they are working on something for them for free. (Which would be some note completion BS stuff, rather than actual clinical support that helps patients and cuts costs). They may be doing this already.
  - coredog64 4y ago
    Training a model on an EHR is worse than nothing. Epic allows infinite customization, and customers build up their own informal standards such that you can’t dump and compare data across multiple sites.
    - 4y ago
SrslyJosh 4y ago
Reminder that these models have no concept of truth, no abstract reasoning abilities, and there's no guarantee that the plausible-looking text they spit out isn't complete bullshit. Their output is impressive considering that they're incapable of reasoning, but it desperately needs to be sanity-checked by someone with relevant knowledge before using it for anything serious.
- EGreg 4y ago
  I think it's "top-down reasoning" It does in fact create some sort of model out of those billions of dimensions, and it's able to re-use it across many different fields, too It's not "bottom-up reasoning" so it misses a bunch of little details. Basically AI is a search through a vast space, and most of the approaches have focused on top-down search. Reasoning bottom-up is what AlphaBeta search did (rybka and all those other chess programs, that calculated all possible combinations 10 moves ahead). Those missed some of the "top-down positional" stuff, whereas AlphaGo missed some weird-ass edge cases https://www.engadget.com/alphago-pushed-human-go-players-to-become-more-creative-231703950.html https://www.engadget.com/alphago-pushed-human-go-players-to-... But it is a sort of intelligence. Just not one you can trust when details matter. You can use it for pretty pictures for now, and essays. Generative AI is not deep but extremely broad. It knows many subjects. And can discuss more than any specific person. But guess who produced all that content for it and made it publicly available. Humans! (And in cases where it wasn't publicly available, I guess they can sue the company)
- KyeRussell 4y ago
  Ironically this “convincing bullshit” comment is turning into something everyone parrots in every LLM discussion thread. I think that someone that’s gone as far to built this with GPT-n is aware of its limitations.
  - jacquesm 4y ago
    The question isn't really whether someone that has built it is aware of the limitation, but whether the users of that thing that someone built is aware of them. In one study (see reference) users were able to identify the AI about 2/3rd of the times (and identical for the human experts). So 1/3rd of the time the AI is not detected and may well be spouting nonsense. https://www.medrxiv.org/content/10.1101/2023.01.23.23284735v2 https://www.medrxiv.org/content/10.1101/2023.01.23.23284735v...
ReptileMan 4y ago
>Better than first year residents - but not necessarily experienced clinicians. The point is to be better than self diagnosis with webmd ...
ergonaught 4y ago
Here’s the fundamental problem. By any reasonable standard, a “clinical pharmacist for 10 years” should understand that there are circumstances where there is a correct answer, and incorrect answers are potentially catastrophic. You, clearly, are either unable to recognize that or are unable to recognize that LLM are dangerously useless in such contexts. Either you already know the correct answer, making this useless, or you are not competent to utilize its answers because you can’t tell whether it’s about to kill someone with plausible nonsense. And if a “clinical pharmacist for 10 years” can lose sight of this really fundamental issue, this whole thing needs to be halted. Governmentally if necessary. And it won’t be. It’s bad enough that “bad actors” will leverage this for their purposes, but the presumably competent simply turning off their brains over this is the most perplexing glitch ever to be on public display.
- aiappreciator 4y ago
  "Either you already know the correct answer, making this useless, or you are not competent to utilize its answers because you can’t tell whether it’s about to kill someone with plausible nonsense" Evaluating if an answer is correct or not is easier than coming up with a correct answer from scratch. NP != P. if you don't understand this basic fact, then you are not competent enough to comment on AI.
  - underanalyzer 4y ago
    A few of the comments in this thread seem to be misusing mathematics in order to lend more credence to themselves. At the risk of responding to low quality flamebait here are some problems with your statements. 1. P = NP refers to a two very specific sets of problems (which might actually be the same set) not any general question. There are problems that we know don't fall into P or NP, (for example the Halting Problem). Also whether or not P=NP is an open question almost the opposite of a fact. 2. You claim: "Evaluating if an answer is correct or not is easier than coming up with a correct answer from scratch." This is the right idea but not quite correct. The correct statement is as follows: "Evaluating if an answer is correct is not harder than the difficulty of coming up with a correct answer from scratch." This is because evaluating some answer can still be just as hard as the original problem. In fact sometimes it's uncomputable (if the original problem is also uncomputable). To use an example from above consider the question: "Does a program x halt?" If I tell you "no" it could be impossible to verify my answer unless you have solved the halting problem. To bring this back to reality, again if GPT-4 is wrong about some complex medical question it doesn't mean it's mathematically easier to figure that out than solving the problem from scratch.
zora_goron 4y ago
GalenAI looks very interesting -- definitely will be taking a closer look! :) >> Now - throw a punch of clinical guidelines in a vector database and give it context I built MedQA (https://labs.cactiml.com/medqa https://labs.cactiml.com/medqa) as a way to explore how GPT could be used through providing clinical guidelines as context + pretty extensive prompt engineering. There are definitely limitations and the accuracy/constraining "hallucinations" has to meet a very high bar, but I've found it interesting-to-helpful several times while on rounds at the hospital as a med student. Some functionality made possible through GPT that I am excited to explore further: - Interacting with the guidelines in a question-and-followup format: https://twitter.com/samarthrawal/status/1620547117717786624 https://twitter.com/samarthrawal/status/1620547117717786624 - Answering with different levels of complexity depending on user: https://twitter.com/samarthrawal/status/1636137740390498306 https://twitter.com/samarthrawal/status/1636137740390498306 - Proactively detecting if the answer lends itself well to a table format, and optionally generating a table to address user query: https://twitter.com/samarthrawal/status/1631759290414276615 https://twitter.com/samarthrawal/status/1631759290414276615
yosito 4y ago
GalenAI is such an awesome idea, and I'm genuinely rooting for its success! I just wanted to point out a small typo in one of the headings: "So how does Galen really works?" might be better as "So how does Galen really work?". Having a native speaker give your copy a quick check could be a helpful way to ensure it comes across as polished and professional. :)
crakenzak 4y ago
> This because there are very niche stuff as you specialize where it’s only like 5 doctors who see it in the whole world on a consistent basis (and they don’t blog!) We need to do a better job with sharing niche information as a society! Imagine all the benefits for medicine that can come from a model knowing about ultra rare and niche occurrences happening all over the world.
djmips 4y ago
And therein lies a problem. In a world with billions of people, or even a country of 300 million - how could someone possibly get diagnosed when their problem needs to be put in front of one of five doctors!

joshgel 4y ago

Ya, internist here. For some context, the USMLE is taken during medical school. The amount I have learned about actually practicing medicine since graduating is probably an order of magnitude more than everything I learned in medical school! I still learn stuff, all the time, and I’m not just talking about new research. So, while impressive and clearly part of the future world, we shouldn’t get too far ahead of ourselves with the current models. Edit: oh I should add that there are more clinically relevant exams that would be more likely to reveal d clinical usefulness, for example “board” exams. These are taken after training, usually before practice. Not knocking LLMs, just ensuring that people don’t misunderstand passing the USMLE as being clinically useful.

capableweb 4y ago
I agree that we shouldn't get ahead of ourselves with the current technology, but what you said earlier applies to practically every industry and science. What you learn at the actual job is always far more up to date than what you learn in school, no matter if it's being a engineer, doctor or just a lowly programmer.
- notch898a 4y ago
  Yes but the difference is most engineers, and pretty much all lowly programmers are unlicensed. AI or some non-human accessible in the countryside so you don't have to order questionable "fish" antibiotics or "cat" anti-parasitic would be a nice step up from the current gatekeeping of medicine from people with limited access.
  - wbl 4y ago
    There are much simpler ways to provide access to those areas.
    - notch898a 4y ago
      The only significant ways I'm aware of that people get needed rx medications outside of physicians and mid-level practitioners is to leave the country or use "vet"/"animal" drugs. What are the simpler other alternatives currently available?
      - wbl 4y ago
        Make more physicians and pay them to work in rural areas. Congress can just do this.
        goldenkey 4y ago
        American Medical Association is basically a union that artificially caps the number of doctors of each speciality.
        notch898a 4y ago
        The medical cartel loves to cloak their policies under the auspices of safety. In the end you'll find their policies magically result in massive profits for bureaucrats and chokepoints that constrain the supply of gatekeepers. This is not accidental.
Madmallard 4y ago
Ive never experienced anything in my care with doctors that I couldn’t understand with a days worth of research into things like UpToDate. It isn’t complicated. It is largely memorization and application of an algorithm which is just borderline useless for complex conditions that are emerging more modernly.
- cguess 4y ago
  Have you ever had anything besides a bad cold? Tell me you understand every acronym in this article, and the ability to explain it succinctly to a patient much less to be able to hand off a case to "another" physician https://www.nejm.org/doi/full/10.1056/NEJMoa2206714 https://www.nejm.org/doi/full/10.1056/NEJMoa2206714 Doctors and nurses have saved me many times from some very close calls because of decades of experience, training and intuition. That is of course not to mention the friend who beat a deep brain tumor on their brain stem that everyone else told them was inoperable, and now are in medical school themselves for neurology. No LLM is going to pull that out of itself, possibly ever, and certainly not GPT-4 (no one else had ever had the surgery done before, it was novel).
  - Madmallard 4y ago
    I have an extremely large amount of experience in this context.
- deleted 4y ago
  [deleted]
- Veen 4y ago
  How do you know that you actually understand a topic as well as a doctor? How do you verify that? It's not unusual for people to think they comprehend a topic at expert level, when in fact they do not. The correlation between confidence and understanding is not a reliable measure. That's why doctors are trained by more expert colleagues who can judge their true understanding, have to take exams, etc.
  - Madmallard 4y ago
sizzle 4y ago
What if the AI is trained on board exams and other high signal testing/examination materials? Surely it will become superhuman in its medical abilities?

TaylorAlexander 4y ago

This makes me think we need some kind of program for experts to start writing things down in a way which is helpful. Even just take dictation and transcribe it.

another_story 4y ago
There are many tools for doctors to do just this, but it's a matter of time more than anything.
andrewthornton 4y ago
You should take a quick look at EPIC. They dominate the electronic heath record space, and a ton of health systems use it. You will know if your doctor's office uses an EHR application, because they will be typing notes into it for the majority of your visit. I have not been too excited about the amount of time that physicians spend on EHR systems, but I am hopeful that taking the data they input (along with blood work and other test results) will make everything more accurate, fast and effective.
- jonathan-adly 4y ago
  EPIC unfortunately is all the bad things about Google, and none of the good. Unable to ship anything, protect their margin > help the users solve problems, monopoly, locked up distribution so no one else can innovate. Honestly, my bear case for AI in medicine is Epic picking up the phone and telling health-systems not to buy anything because they are working on something for them for free. (Which would be some note completion BS stuff, rather than actual clinical support that helps patients and cuts costs). They may be doing this already.
- coredog64 4y ago
  Training a model on an EHR is worse than nothing. Epic allows infinite customization, and customers build up their own informal standards such that you can’t dump and compare data across multiple sites.
  - 4y ago

SrslyJosh 4y ago

Reminder that these models have no concept of truth, no abstract reasoning abilities, and there's no guarantee that the plausible-looking text they spit out isn't complete bullshit. Their output is impressive considering that they're incapable of reasoning, but it desperately needs to be sanity-checked by someone with relevant knowledge before using it for anything serious.

EGreg 4y ago
I think it's "top-down reasoning" It does in fact create some sort of model out of those billions of dimensions, and it's able to re-use it across many different fields, too It's not "bottom-up reasoning" so it misses a bunch of little details. Basically AI is a search through a vast space, and most of the approaches have focused on top-down search. Reasoning bottom-up is what AlphaBeta search did (rybka and all those other chess programs, that calculated all possible combinations 10 moves ahead). Those missed some of the "top-down positional" stuff, whereas AlphaGo missed some weird-ass edge cases https://www.engadget.com/alphago-pushed-human-go-players-to-become-more-creative-231703950.html https://www.engadget.com/alphago-pushed-human-go-players-to-... But it is a sort of intelligence. Just not one you can trust when details matter. You can use it for pretty pictures for now, and essays. Generative AI is not deep but extremely broad. It knows many subjects. And can discuss more than any specific person. But guess who produced all that content for it and made it publicly available. Humans! (And in cases where it wasn't publicly available, I guess they can sue the company)
KyeRussell 4y ago
Ironically this “convincing bullshit” comment is turning into something everyone parrots in every LLM discussion thread. I think that someone that’s gone as far to built this with GPT-n is aware of its limitations.
- jacquesm 4y ago
  The question isn't really whether someone that has built it is aware of the limitation, but whether the users of that thing that someone built is aware of them. In one study (see reference) users were able to identify the AI about 2/3rd of the times (and identical for the human experts). So 1/3rd of the time the AI is not detected and may well be spouting nonsense. https://www.medrxiv.org/content/10.1101/2023.01.23.23284735v2 https://www.medrxiv.org/content/10.1101/2023.01.23.23284735v...

ReptileMan 4y ago

>Better than first year residents - but not necessarily experienced clinicians. The point is to be better than self diagnosis with webmd ...

ergonaught 4y ago

Here’s the fundamental problem. By any reasonable standard, a “clinical pharmacist for 10 years” should understand that there are circumstances where there is a correct answer, and incorrect answers are potentially catastrophic. You, clearly, are either unable to recognize that or are unable to recognize that LLM are dangerously useless in such contexts. Either you already know the correct answer, making this useless, or you are not competent to utilize its answers because you can’t tell whether it’s about to kill someone with plausible nonsense. And if a “clinical pharmacist for 10 years” can lose sight of this really fundamental issue, this whole thing needs to be halted. Governmentally if necessary. And it won’t be. It’s bad enough that “bad actors” will leverage this for their purposes, but the presumably competent simply turning off their brains over this is the most perplexing glitch ever to be on public display.

aiappreciator 4y ago
"Either you already know the correct answer, making this useless, or you are not competent to utilize its answers because you can’t tell whether it’s about to kill someone with plausible nonsense" Evaluating if an answer is correct or not is easier than coming up with a correct answer from scratch. NP != P. if you don't understand this basic fact, then you are not competent enough to comment on AI.
- underanalyzer 4y ago
  A few of the comments in this thread seem to be misusing mathematics in order to lend more credence to themselves. At the risk of responding to low quality flamebait here are some problems with your statements. 1. P = NP refers to a two very specific sets of problems (which might actually be the same set) not any general question. There are problems that we know don't fall into P or NP, (for example the Halting Problem). Also whether or not P=NP is an open question almost the opposite of a fact. 2. You claim: "Evaluating if an answer is correct or not is easier than coming up with a correct answer from scratch." This is the right idea but not quite correct. The correct statement is as follows: "Evaluating if an answer is correct is not harder than the difficulty of coming up with a correct answer from scratch." This is because evaluating some answer can still be just as hard as the original problem. In fact sometimes it's uncomputable (if the original problem is also uncomputable). To use an example from above consider the question: "Does a program x halt?" If I tell you "no" it could be impossible to verify my answer unless you have solved the halting problem. To bring this back to reality, again if GPT-4 is wrong about some complex medical question it doesn't mean it's mathematically easier to figure that out than solving the problem from scratch.

zora_goron 4y ago

GalenAI looks very interesting -- definitely will be taking a closer look! :) >> Now - throw a punch of clinical guidelines in a vector database and give it context I built MedQA (https://labs.cactiml.com/medqa https://labs.cactiml.com/medqa) as a way to explore how GPT could be used through providing clinical guidelines as context + pretty extensive prompt engineering. There are definitely limitations and the accuracy/constraining "hallucinations" has to meet a very high bar, but I've found it interesting-to-helpful several times while on rounds at the hospital as a med student. Some functionality made possible through GPT that I am excited to explore further: - Interacting with the guidelines in a question-and-followup format: https://twitter.com/samarthrawal/status/1620547117717786624 https://twitter.com/samarthrawal/status/1620547117717786624 - Answering with different levels of complexity depending on user: https://twitter.com/samarthrawal/status/1636137740390498306 https://twitter.com/samarthrawal/status/1636137740390498306 - Proactively detecting if the answer lends itself well to a table format, and optionally generating a table to address user query: https://twitter.com/samarthrawal/status/1631759290414276615 https://twitter.com/samarthrawal/status/1631759290414276615

yosito 4y ago

GalenAI is such an awesome idea, and I'm genuinely rooting for its success! I just wanted to point out a small typo in one of the headings: "So how does Galen really works?" might be better as "So how does Galen really work?". Having a native speaker give your copy a quick check could be a helpful way to ensure it comes across as polished and professional. :)

crakenzak 4y ago

> This because there are very niche stuff as you specialize where it’s only like 5 doctors who see it in the whole world on a consistent basis (and they don’t blog!) We need to do a better job with sharing niche information as a society! Imagine all the benefits for medicine that can come from a model knowing about ultra rare and niche occurrences happening all over the world.

djmips 4y ago

And therein lies a problem. In a world with billions of people, or even a country of 300 million - how could someone possibly get diagnosed when their problem needs to be put in front of one of five doctors!