4 ms·

Claude and LLaMa get basic human anatomy wrong

12 points by johntiger1 3y ago

amilios 3y ago
I mean, it's a well known fact that LLMs do badly in the "long tail" of the distribution: anything they haven't seen many many times (e.g. relatively obscure specialist knowledge), they are definitely very likely to get wrong. The more "popular" the thing you're asking from them is, the more likely it is they'll answer correctly.
- johntiger1 3y ago
  I don't think it's that obscure. It's something taught in first-year medical school, or in undergrad anatomy or physiology. Hardly anything that deep
  - haldujai 3y ago
    Really? It's not even the most common way to refer to the caval foramen. This is obscure archaic terminology. It's caval foramen, esophageal hiatus and aortic hiatus. Clinically, everyone says opening, foramen or aperture.
ianbutler 3y ago
I disagree with the idea that this is basic human anatomy, basic at the level of a practitioner, likely, basic at the level of a lay person like myself? Not certain. Had you asked me "Hey Ian, at what level is the caval hiatus?" I would either tell you I had no clue or look it up. If anything this demonstrates that LLMs aren't great with niche knowledge, which makes sense since the training corpuses are constructed from common knowledge for the most part. This is however, a potentially great case for retrieval augmented generation.
- johntiger1 3y ago
  Interestingly, we actually do exactly that, although our underlying LLM is OpenAI-based, so we would have avoided the error in the first place :) > I disagree with the idea that this is basic human anatomy When I say basic, I mean something taught in second or third year anatomy or physiology courses; or first year medical school.
  - ianbutler 3y ago
    Fair enough, It's still more specialized knowledge and I'd guess Anthropic's training corpus is light on it. If I recall correctly OpenAI included a lot of textbook knowledge.
    - haldujai 3y ago
      No you're right. He's using an incredibly obscure term (caval hiatus rather than caval opening/foramen/aperture) for a mild-moderately obscure question. Nevertheless I would wager most clinicians (except for surgeons, radiologists or cardiologists) wouldn't know the answer to this question. You learn this in first year of medical school and then forget it.
  - haldujai 3y ago
    Out of curiosity what stage are you in your medical training? You will never hear "caval hiatus" after pre-clerkship.
orliesaurus 3y ago
Why Claude and LLaMa get it wrong but OpenAI's LLM get it right?
- johntiger1 3y ago
  Perhaps more time with RLHF due to first mover advantage? If you figure it out, Meta and Anthropic would probably pay big $$ for the secret
dweekly 3y ago
FWIW, Pi returns: "The caval hiatus is located at the level of the T8 vertebra, which is near the middle of the spine. It's a small opening in the diaphragm that allows the inferior vena cava, a major vein, to pass through. There's also a nearby opening called the esophageal hiatus, which is where the esophagus passes through the diaphragm. The caval hiatus is an important anatomical landmark, and it's also sometimes called the caval opening or the caval orifice. Is this something you're studying for a class or just out of curiosity?"
- johntiger1 3y ago
  Interesting, haven't heard of Pi. Is this a custom LLM?
  - jrflowers 3y ago
    [flagged]
carbocation 3y ago
As a cardiologist I will translate this into software: saying that the name of the portion of the diaphgragm that opens to the IVC is "basic human anatomy" is like saying that "monoids in the category of endofunctors" is a basic tenet of software development.
- johntiger1 3y ago
  Don't think I learned about monoids in 1st year CS. I have learned "I ate 10 eggs at 12" in 1st year medical school.
  - yCombLinks 3y ago
    Your first year of medical school is after your 4 years of undergrad.
    - johntiger1 3y ago
      Not in Quebec or the UK.
  - carbocation 3y ago
    And do you think that “caval hiatus,” the phrase that you used in your prompts, is the term most likely to elicit that from LLMs? I ask because using plain medical language seems to elicit the correct answer from Claude and ChatGPT3.5 (I didn’t test llama from my phone.)
    - haldujai 3y ago
      As a radiologist who looks at thoracoabdominal anatomy for a living I completely agree. That was also my initial reaction, I don't think I've ever read "caval hiatus" used in clinical text other than maybe during medical school. It's not even correct, anatomically it's most correctly described as the caval foramen. Hiatus refers to a slit/gap, foramen refers to a circumscribed orifice. Caval hiatus is either misremembering or an archaic text. Clinically people use aperture, foramen or opening. Other problems with this post: I assume the incredibly poor way of prompting is attempting to simulate a user using a chat interface? If so, are they even using the right LLaMA model (at the very least chat fine-tuned, ideally also on the open medical conversation dataset)? What GPT model are we comparing with? Is there system context provided for the prompt? If you were deploying this you wouldn't only pass the inputed user text in the prompt. ChatGPT's interface adds this so if you're going to make a fair comparison you should be doing the same. It's still going to favor OpenAI for this specific use case (vague and poorly phrased short prompts of esoteric trivia) as they've had much more training in this domain.
nperez 3y ago
ChatGPT uses a 1.7 trillion parameter model IIRC. I wouldn't expect a 70b model to compare well. There are medical finetunes that might be better to try for this, like https://huggingface.co/wanglab/ClinicalCamel-70B https://huggingface.co/wanglab/ClinicalCamel-70B Edit/Correction: as mentioned below, free tier used by OP is in the neighborhood of 175b
- johntiger1 3y ago
  Where are you getting the 1.7T number from? Genuinely curious, as sources like [1] generally cite 175B. [1]: https://ecoagi.ai/articles/chatgpt-parameters https://ecoagi.ai/articles/chatgpt-parameters
  - nperez 3y ago
    I'm not sure, I've tried looking it up and couldn't find an official answer. But that site says GPT3 is 175b and GPT4 is 170 trillion. That's insane if true Edit: and this one says 1.7, idk https://levelup.gitconnected.com/gpt-4-parameters-explained-everything-you-need-to-know-e210c20576ca https://levelup.gitconnected.com/gpt-4-parameters-explained-... Haven't used chatgpt in a bit, but looks like the free tier is GPT3 still, so 175b may be correct if OP isn't a subscriber
    - johntiger1 3y ago
      I was testing on GPT3.5-Turbo, the free tier.
      - nperez 3y ago
        Ah, 175b it is then. That's actually kind of encouraging that an open source trillion+ param model isn't needed to get there. A 180b Falcon model very recently came out https://huggingface.co/tiiuae/falcon-180B https://huggingface.co/tiiuae/falcon-180B A finetune of this might be competitive, but I haven't tried it.
      - nickthegreek 3y ago
tayo42 3y ago
I wonder if it would do better with a better prompt?

12 points by johntiger1 3y ago

I mean, it's a well known fact that LLMs do badly in the "long tail" of the distribution: anything they haven't seen many many times (e.g. relatively obscure specialist knowledge), they are definitely very likely to get wrong. The more "popular" the thing you're asking from them is, the more likely it is they'll answer correctly.

johntiger1 3y ago
I don't think it's that obscure. It's something taught in first-year medical school, or in undergrad anatomy or physiology. Hardly anything that deep
- haldujai 3y ago
  Really? It's not even the most common way to refer to the caval foramen. This is obscure archaic terminology. It's caval foramen, esophageal hiatus and aortic hiatus. Clinically, everyone says opening, foramen or aperture.

ianbutler 3y ago

I disagree with the idea that this is basic human anatomy, basic at the level of a practitioner, likely, basic at the level of a lay person like myself? Not certain. Had you asked me "Hey Ian, at what level is the caval hiatus?" I would either tell you I had no clue or look it up. If anything this demonstrates that LLMs aren't great with niche knowledge, which makes sense since the training corpuses are constructed from common knowledge for the most part. This is however, a potentially great case for retrieval augmented generation.

johntiger1 3y ago
Interestingly, we actually do exactly that, although our underlying LLM is OpenAI-based, so we would have avoided the error in the first place :) > I disagree with the idea that this is basic human anatomy When I say basic, I mean something taught in second or third year anatomy or physiology courses; or first year medical school.
- ianbutler 3y ago
  Fair enough, It's still more specialized knowledge and I'd guess Anthropic's training corpus is light on it. If I recall correctly OpenAI included a lot of textbook knowledge.
  - haldujai 3y ago
    No you're right. He's using an incredibly obscure term (caval hiatus rather than caval opening/foramen/aperture) for a mild-moderately obscure question. Nevertheless I would wager most clinicians (except for surgeons, radiologists or cardiologists) wouldn't know the answer to this question. You learn this in first year of medical school and then forget it.
- haldujai 3y ago
  Out of curiosity what stage are you in your medical training? You will never hear "caval hiatus" after pre-clerkship.

orliesaurus 3y ago

Why Claude and LLaMa get it wrong but OpenAI's LLM get it right?

johntiger1 3y ago
Perhaps more time with RLHF due to first mover advantage? If you figure it out, Meta and Anthropic would probably pay big $$ for the secret

dweekly 3y ago

FWIW, Pi returns: "The caval hiatus is located at the level of the T8 vertebra, which is near the middle of the spine. It's a small opening in the diaphragm that allows the inferior vena cava, a major vein, to pass through. There's also a nearby opening called the esophageal hiatus, which is where the esophagus passes through the diaphragm. The caval hiatus is an important anatomical landmark, and it's also sometimes called the caval opening or the caval orifice. Is this something you're studying for a class or just out of curiosity?"

johntiger1 3y ago
Interesting, haven't heard of Pi. Is this a custom LLM?
- jrflowers 3y ago
  [flagged]

carbocation 3y ago

As a cardiologist I will translate this into software: saying that the name of the portion of the diaphgragm that opens to the IVC is "basic human anatomy" is like saying that "monoids in the category of endofunctors" is a basic tenet of software development.

johntiger1 3y ago
Don't think I learned about monoids in 1st year CS. I have learned "I ate 10 eggs at 12" in 1st year medical school.
- yCombLinks 3y ago
  Your first year of medical school is after your 4 years of undergrad.
  - johntiger1 3y ago
    Not in Quebec or the UK.
- carbocation 3y ago
  And do you think that “caval hiatus,” the phrase that you used in your prompts, is the term most likely to elicit that from LLMs? I ask because using plain medical language seems to elicit the correct answer from Claude and ChatGPT3.5 (I didn’t test llama from my phone.)
  - haldujai 3y ago
    As a radiologist who looks at thoracoabdominal anatomy for a living I completely agree. That was also my initial reaction, I don't think I've ever read "caval hiatus" used in clinical text other than maybe during medical school. It's not even correct, anatomically it's most correctly described as the caval foramen. Hiatus refers to a slit/gap, foramen refers to a circumscribed orifice. Caval hiatus is either misremembering or an archaic text. Clinically people use aperture, foramen or opening. Other problems with this post: I assume the incredibly poor way of prompting is attempting to simulate a user using a chat interface? If so, are they even using the right LLaMA model (at the very least chat fine-tuned, ideally also on the open medical conversation dataset)? What GPT model are we comparing with? Is there system context provided for the prompt? If you were deploying this you wouldn't only pass the inputed user text in the prompt. ChatGPT's interface adds this so if you're going to make a fair comparison you should be doing the same. It's still going to favor OpenAI for this specific use case (vague and poorly phrased short prompts of esoteric trivia) as they've had much more training in this domain.

nperez 3y ago

ChatGPT uses a 1.7 trillion parameter model IIRC. I wouldn't expect a 70b model to compare well. There are medical finetunes that might be better to try for this, like https://huggingface.co/wanglab/ClinicalCamel-70B https://huggingface.co/wanglab/ClinicalCamel-70B Edit/Correction: as mentioned below, free tier used by OP is in the neighborhood of 175b

johntiger1 3y ago
Where are you getting the 1.7T number from? Genuinely curious, as sources like [1] generally cite 175B. [1]: https://ecoagi.ai/articles/chatgpt-parameters https://ecoagi.ai/articles/chatgpt-parameters
- nperez 3y ago
  I'm not sure, I've tried looking it up and couldn't find an official answer. But that site says GPT3 is 175b and GPT4 is 170 trillion. That's insane if true Edit: and this one says 1.7, idk https://levelup.gitconnected.com/gpt-4-parameters-explained-everything-you-need-to-know-e210c20576ca https://levelup.gitconnected.com/gpt-4-parameters-explained-... Haven't used chatgpt in a bit, but looks like the free tier is GPT3 still, so 175b may be correct if OP isn't a subscriber
  - johntiger1 3y ago
    I was testing on GPT3.5-Turbo, the free tier.
    - nperez 3y ago
      Ah, 175b it is then. That's actually kind of encouraging that an open source trillion+ param model isn't needed to get there. A 180b Falcon model very recently came out https://huggingface.co/tiiuae/falcon-180B https://huggingface.co/tiiuae/falcon-180B A finetune of this might be competitive, but I haven't tried it.
    - nickthegreek 3y ago

tayo42 3y ago

I wonder if it would do better with a better prompt?