4 ms·

There's a famous story (probably apocryphal) about the military of a powerful nation training an early AI to find pictures of submarines beneath the sea. There

There's a famous story (probably apocryphal) about the military of a powerful nation training an early AI to find pictures of submarines beneath the sea. There was great excitement as it was near 100%. It later transpired the pictures with submarines in had a white border.

morkalork 3y ago
I've heard this one before but with soviet tanks back in the 2000s!
- erehweb 3y ago
  May be an urban legend, according to this site: https://gwern.net/tank https://gwern.net/tank
- sva_ 3y ago
  I think there was something about an ANN detecting skin melanoma, assigning a high likelihood if there is a clinic/doctors office in the background.
sterlind 3y ago
haven't modern model architectures gotten better at avoiding this kind of overfitting? like obviously data quality is still very important, but my understanding is that dropout mitigates this by randomly cutting out these unwanted feature channels. the models learn to distinguish all differences, rather than just one, or fixed combinations of several.
- dragonwriter 3y ago
  > haven't modern model architectures gotten better at avoiding this kind of overfitting? Overfitting is, AIUI, a training method and data issue, not a model issue alone. I doubt any model is resistant to overfitting if you give it data where the answer is reliably encoded some aspect it can use but outside of what you want it to look at. Now, you can notice suspicious results and investigate (or you can just publish a 100% success rate and call it a day.)
- daveguy 3y ago
  It really doesn't have to do with most ML architectures. It has to do with experiment design. If some data used in testing is part of the training process there will be over fitting. That's why a final test set is required for unbiased evaluation.
MrMember 3y ago
Ha, I've heard a similar story about diagnosing skin cancer from pictures of moles. They were really excited about the performance of the model but it turned out if the dermatologist was concerned about the size of a mole they would include a ruler in the picture to document the size. The NN wasn't trained to "diagnose skin cancer" it was trained to recognize rulers in pictures.

morkalork 3y ago

I've heard this one before but with soviet tanks back in the 2000s!

erehweb 3y ago
May be an urban legend, according to this site: https://gwern.net/tank https://gwern.net/tank
sva_ 3y ago
I think there was something about an ANN detecting skin melanoma, assigning a high likelihood if there is a clinic/doctors office in the background.

sterlind 3y ago

haven't modern model architectures gotten better at avoiding this kind of overfitting? like obviously data quality is still very important, but my understanding is that dropout mitigates this by randomly cutting out these unwanted feature channels. the models learn to distinguish all differences, rather than just one, or fixed combinations of several.

dragonwriter 3y ago
> haven't modern model architectures gotten better at avoiding this kind of overfitting? Overfitting is, AIUI, a training method and data issue, not a model issue alone. I doubt any model is resistant to overfitting if you give it data where the answer is reliably encoded some aspect it can use but outside of what you want it to look at. Now, you can notice suspicious results and investigate (or you can just publish a 100% success rate and call it a day.)
daveguy 3y ago
It really doesn't have to do with most ML architectures. It has to do with experiment design. If some data used in testing is part of the training process there will be over fitting. That's why a final test set is required for unbiased evaluation.

MrMember 3y ago

Ha, I've heard a similar story about diagnosing skin cancer from pictures of moles. They were really excited about the performance of the model but it turned out if the dermatologist was concerned about the size of a mole they would include a ruler in the picture to document the size. The NN wasn't trained to "diagnose skin cancer" it was trained to recognize rulers in pictures.