5 ms·
“Just review the AI draft” is likely gonna kill some people by priming clinicians to agree with the result. https://www.npr.org/sections/health-shots/2013/02/1
by ceejayoz 3y ago
“Just review the AI draft” is likely gonna kill some people by priming clinicians to agree with the result.
https://www.npr.org/sections/health-shots/2013/02/11/171409656/why-even-radiologists-can-miss-a-gorilla-hiding-in-plain-sight https://www.npr.org/sections/health-shots/2013/02/11/1714096...
> He took a picture of a man in a gorilla suit shaking his fist, and he superimposed that image on a series of slides that radiologists typically look at when they're searching for cancer. He then asked a bunch of radiologists to review the slides of lungs for cancerous nodules. He wanted to see if they would notice a gorilla the size of a matchbook glaring angrily at them from inside the slide.
> But they didn't: 83 percent of the radiologists missed it, Drew says.
- wccrawford 3y agoBased on the code that CoPilot has produced for me, I can 100% see that happening, and rather a lot. So far, the code it produces looks really good at first glance, but once I really dig into what it's doing, I find numerous problems. I have to really think about that code to overcome the things it does wrong. I think many doctors are used to doing that already, since they have to correct their interns/assistants/etc when they come up with things that sound good, but ultimately aren't. But it's definitely something that they'll fail at sometimes.
- stouset 3y agoThis is what I really worry about in programming contexts. The norm is already for a distressing proportion of engineers to barely (if even that) understand code they’re working on, even when they’re the only author. In that situation it’s extremely easy to just blindly trust an “expert” third party, particularly one where you can’t ask it questions about how and why.
- userbinator 3y agoThey don't even deserve to be called "engineers". They're just guess-and-test amateurs.
- userbinator 3y agoFrom what I've seen so far, some will absolutely notice and complain loudly about the mistakes that AI makes, and some won't --- praising it as another great solution. I suspect this will create another highly divisive issue.
- gretch 3y agoIf you put an ascii art comment block of a gorilla in a code base and told me to “find the race condition”, I would probably skip over the gorilla. At the end of the day the metric of success for these doctors is to find cancer, not impossible images of gorillas. The study is kinda “funny”, but it doesn’t really mean anything. Even the article itself does not claim that this is some kind of failure
- ceejayoz 3y agoThe point of this study is that if you’re told there’s a race condition in the code, you are much more likely to miss a big security hole in it.
- gretch 3y agoSo give the a reviewer a rubric of things you care about. Race conditions, security, etc. The point is, an image of a gorilla is literally completely impossible and has no relevance. That’s why a security vuln is not analogous
- ceejayoz 3y agoIn the case of radiology, that rubric is "anything clinically signifcant you can find". A kid in being treated for possible pneumonia might have signs on a chest x-ray of cancer, rib fractures from child abuse, an enlarged heart, etc. There's a risk to "rule out pneumonia" style guidance resulting in a report that misses these things in favor of "yep, it's pneumonia".
- vrc 3y agoA better study would be to place an unlikely but detection worthy artifact into the image. Say, you’re looking for a broken bone and you put a rare disease in an unrelated part of the body but visible in the radiograph. Bet the clinicians spot it. Because in the real world, this is how things get caught. But I’d love to know the missed detection rate.
- 3y ago
- Zenzero 3y agoWith all due respect there is so much absurdity in the assumptions being made with your linked article that it is almost not worth engaging with. However, I will for educational purposes. As someone who is trained in and comfortable reading radiographs but is not a radiologist, I can tell you that putting a gorilla on one of the views is a poor measure of how many things are missed by radiologists. Effectively interpreting imaging studies requires expert knowledge of the anatomy being imaged and the variety of ways pathology is reflected in a visibly detectable manner. What they are doing is rapidly cycling through what is effectively a long checklist of areas to note: evaluate the appearance of hilar and mediastinal lymph nodes, note bronchiolar appearance, is there evidence of interstitial or alveolar patterns (considered within the context of what would be expected for a variety of etiologies such as bronchopneumonia, neoplasia, CHF,...), do you observe appropriate dimensions of the cardiac silhouette, do you see other evidence of consolidation within the lungs, within the visible vertebrae do you observe appropriate alignment, do the endplates appear abnormal, do you observe any vertebral lucencies, on and on and on. Atypical changes are typically clustered in expected ways. Often deviations from what is expected will trigger a bit more consideration, but those expectations are subverted during the course of going through your "checklist". No radiologist has look for a gorilla in their evaluation. It is pretty clear that the layperson's understanding of a radiologist being "look at the picture and say anything that is different" is a complete miss on what is actually happening during their evaluation. It's like if I asked you to show me your skills driving a car around an obstacle course, and then afterwards I said you are a bad driver because you forgot to check that I swapped out one of the lug nuts on each wheel with a Vienna sausage.
- ceejayoz 3y agoMy dad is a radiologist and… not so dismissive of this study. Missing other conditions on a reading due to a focus on something specific is not uncommon. Things like obvious fractures left out of a report. https://cognitiveresearchjournal.springeropen.com/articles/10.1186/s41235-021-00318-w https://cognitiveresearchjournal.springeropen.com/articles/1... > For over 50 years, the satisfaction of search effect has been studied within the field of radiology. Defined as a decrease in detection rates for a subsequent target when an initial target is found within the image, these multiple target errors are known to underlie errors of omission (e.g., a radiologist is more likely to miss an abnormality if another abnormality is identified). More recently, they have also been found to underlie lab-based search errors in cognitive science experiments (e.g., an observer is more likely to miss a target ‘T’ if a different target ‘T’ was detected). This phenomenon was renamed the subsequent search miss (SSM) effect in cognitive science.