5 ms·

I’m shocked that nobody has mentioned this yet: AI intended to recognize skin cancer instead learned to identify doctors’ markings and rulers. https://jamanet

• 4rt 3y ago
  I heard a similar story about AI being used on MRI scans. What the researchers hadn't realised was that the samples were coming from two different models of MRI machine and that patients with more serious symptoms / worse expected outcomes were more often being sent to be scanned on the more advanced of the two. The AI had just detected some hidden difference between the outputs and was infering that the patients scanned on the expensive machine were more likely to have a serious condition.
• divergencefree 3y ago
  Yea this is what I was thinking of. Another work in that direction: https://www.nature.com/articles/s42256-021-00338-7 https://www.nature.com/articles/s42256-021-00338-7 I never heard of the tanks story, but I vaguely remember stuff in radiology which was a similar story basically, it learned some spurious correlation to the label.
• quickthrower2 3y ago
  "This study suggests that the use of surgical skin markers should be avoided in dermoscopic images intended for analysis by a convolutional neural network." Or you train with images where those markers are already removed by another NN? Or use some photos without cancer but with markers in the mix.