4 ms·
I disagree with the idea that this is basic human anatomy, basic at the level of a practitioner, likely, basic at the level of a lay person like myself? Not cer
by ianbutler 3y ago
I disagree with the idea that this is basic human anatomy, basic at the level of a practitioner, likely, basic at the level of a lay person like myself? Not certain. Had you asked me "Hey Ian, at what level is the caval hiatus?" I would either tell you I had no clue or look it up.
If anything this demonstrates that LLMs aren't great with niche knowledge, which makes sense since the training corpuses are constructed from common knowledge for the most part.
This is however, a potentially great case for retrieval augmented generation.
- johntiger1 3y agoInterestingly, we actually do exactly that, although our underlying LLM is OpenAI-based, so we would have avoided the error in the first place :) > I disagree with the idea that this is basic human anatomy When I say basic, I mean something taught in second or third year anatomy or physiology courses; or first year medical school.
- ianbutler 3y agoFair enough, It's still more specialized knowledge and I'd guess Anthropic's training corpus is light on it. If I recall correctly OpenAI included a lot of textbook knowledge.
- haldujai 3y agoNo you're right. He's using an incredibly obscure term (caval hiatus rather than caval opening/foramen/aperture) for a mild-moderately obscure question. Nevertheless I would wager most clinicians (except for surgeons, radiologists or cardiologists) wouldn't know the answer to this question. You learn this in first year of medical school and then forget it.
- haldujai 3y agoOut of curiosity what stage are you in your medical training? You will never hear "caval hiatus" after pre-clerkship.