3 ms·
This study found that an LLM outperformed doctors "on a standardized rubric of diagnostic performance based on differential diagnosis accuracy, appropriateness
by istjohn 1y ago
This study found that an LLM outperformed doctors "on a standardized rubric of diagnostic performance based on differential diagnosis accuracy, appropriateness of supporting and opposing factors, and next diagnostic evaluation steps, validated and graded via blinded expert consensus."
https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2825395 https://jamanetwork.com/journals/jamanetworkopen/fullarticle...
- pingou 1y agoThis study is about doctors using an LLM and it doesn't seem like it made them significantly more accurate than doctors not using LLM.
- roenxi 1y agoIf you look in the discussion section you'll find that wasn't exactly what the study ended up with. I'm looking at the paragraph starting: > An unexpected secondary result was that the LLM alone performed significantly better than both groups of humans, similar to a recent study with different LLM technology. They suspected that the clinicians were not prompting it right since the LLM without humans was observed to be outperforming the LLM with skilled operators.