3 ms·

If you put an ascii art comment block of a gorilla in a code base and told me to “find the race condition”, I would probably skip over the gorilla. At the end

by gretch 3y ago

If you put an ascii art comment block of a gorilla in a code base and told me to “find the race condition”, I would probably skip over the gorilla. At the end of the day the metric of success for these doctors is to find cancer, not impossible images of gorillas. The study is kinda “funny”, but it doesn’t really mean anything. Even the article itself does not claim that this is some kind of failure

ceejayoz 3y ago
The point of this study is that if you’re told there’s a race condition in the code, you are much more likely to miss a big security hole in it.
- gretch 3y ago
  So give the a reviewer a rubric of things you care about. Race conditions, security, etc. The point is, an image of a gorilla is literally completely impossible and has no relevance. That’s why a security vuln is not analogous
  - ceejayoz 3y ago
    In the case of radiology, that rubric is "anything clinically signifcant you can find". A kid in being treated for possible pneumonia might have signs on a chest x-ray of cancer, rib fractures from child abuse, an enlarged heart, etc. There's a risk to "rule out pneumonia" style guidance resulting in a report that misses these things in favor of "yep, it's pneumonia".
- vrc 3y ago
  A better study would be to place an unlikely but detection worthy artifact into the image. Say, you’re looking for a broken bone and you put a rare disease in an unrelated part of the body but visible in the radiograph. Bet the clinicians spot it. Because in the real world, this is how things get caught. But I’d love to know the missed detection rate.
  - ceejayoz 3y ago
    This has been studied in various ways. Lots of perception effects documented. For example: https://cognitiveresearchjournal.springeropen.com/articles/10.1186/s41235-021-00318-w https://cognitiveresearchjournal.springeropen.com/articles/1... > For over 50 years, the satisfaction of search effect has been studied within the field of radiology. Defined as a decrease in detection rates for a subsequent target when an initial target is found within the image, these multiple target errors are known to underlie errors of omission (e.g., a radiologist is more likely to miss an abnormality if another abnormality is identified). More recently, they have also been found to underlie lab-based search errors in cognitive science experiments (e.g., an observer is more likely to miss a target ‘T’ if a different target ‘T’ was detected). This phenomenon was renamed the subsequent search miss (SSM) effect in cognitive science.

by gretch 3y ago

ceejayoz 3y ago

The point of this study is that if you’re told there’s a race condition in the code, you are much more likely to miss a big security hole in it.

gretch 3y ago
So give the a reviewer a rubric of things you care about. Race conditions, security, etc. The point is, an image of a gorilla is literally completely impossible and has no relevance. That’s why a security vuln is not analogous
- ceejayoz 3y ago
  In the case of radiology, that rubric is "anything clinically signifcant you can find". A kid in being treated for possible pneumonia might have signs on a chest x-ray of cancer, rib fractures from child abuse, an enlarged heart, etc. There's a risk to "rule out pneumonia" style guidance resulting in a report that misses these things in favor of "yep, it's pneumonia".
vrc 3y ago
A better study would be to place an unlikely but detection worthy artifact into the image. Say, you’re looking for a broken bone and you put a rare disease in an unrelated part of the body but visible in the radiograph. Bet the clinicians spot it. Because in the real world, this is how things get caught. But I’d love to know the missed detection rate.
- ceejayoz 3y ago
  This has been studied in various ways. Lots of perception effects documented. For example: https://cognitiveresearchjournal.springeropen.com/articles/10.1186/s41235-021-00318-w https://cognitiveresearchjournal.springeropen.com/articles/1... > For over 50 years, the satisfaction of search effect has been studied within the field of radiology. Defined as a decrease in detection rates for a subsequent target when an initial target is found within the image, these multiple target errors are known to underlie errors of omission (e.g., a radiologist is more likely to miss an abnormality if another abnormality is identified). More recently, they have also been found to underlie lab-based search errors in cognitive science experiments (e.g., an observer is more likely to miss a target ‘T’ if a different target ‘T’ was detected). This phenomenon was renamed the subsequent search miss (SSM) effect in cognitive science.