5 ms·
Yikes, do I read this correctly and the LLM alone outperforms clinician + LLM?
by guender 3y ago
Yikes, do I read this correctly and the LLM alone outperforms clinician + LLM?
- lucubratory 3y agoYeah, it's a bit awkward.
- ugh123 3y agoIt also outperforms clinician + googling which the study observed (many doctors use google search to help with diagnosis, terminology, recent studies, etc)
- chrstfer 3y agoHell, the LLMs themselves outperformed clinician plus LLM...
- fnordpiglet 3y agoThis has come up before in similar contexts. Model based decision decisions tend to do better than clinicians alone or clinicians with the model based decision system in general. Despite this it’s been effectively impossible to deploy such systems in a clinical setting due to many issues but one is that clinicians aren’t willing to cede their ground and patients aren’t willing to believe the machine over a human.
- caycep 3y agogranted, I feel like the training of some physicians also doesn't make them that good at diagnosis. Over time, intuitively understanding the narrative history of the illness/symptoms is key; that being said, nowadays, this sort of thing seems scattershot in some younger physicians. Med schools tend to try and rejigger their curriculum frequently to try and justifiably make the experience friendlier and w/ less workload for the students, but sometimes, it backfires in terms of this. I had an older school professor who liked to remark, in his day, no such thing as a differential diagnosis, just the right one and a bunch of wrong ones.
- 0xDEAFBEAD 3y agoIs there any way for me to get ahold of one of these models for my own private use?
- rafaelero 3y agoI don't see how those models couldn't be used to at least offer a second opinion. They are pretty cheap to run. I doubt patients would complain about that.
- rafaelero 3y agoThat surprised me as well. The case for humans being merely assisted by AI may not be the strongest one. It could be that they work better without human input, which is at the same time amazing and a bit frightening.
- Terr_ 3y agoJust to make analysis more complicated, not all errors in are created equal: It may be better to have lots of safe errors versus a few huge ones.
- kelseyfrog 3y agoIs that across all subspecialties? I hear internists talking about +LL and -LL all the time and I'd hope that reflected some higher degree of rationality when reasoning out diagnoses, but perhaps not?
- twobitshifter 3y agoMakes sense. In many applications the ‘human in the loop’ will become the weakest link. Imagine giving alpha go to players as a suggested move assistant. Even the best players would have seen alpha go’s suggestions and thought they were an error. Can we eventually trust AI even if we can’t understand its reasoning?
- chrstfer 3y agoThey'll be able to explain their reasoning......
- twobitshifter 3y agoExplaining and understanding the explanation are different.