4 ms·
FWIW, feature maps are not really reliable (to put it mildly). Knowing what area the model is "looking at" is virtually meaningless if you don't know what it's
by andy99 3y ago
FWIW, feature maps are not really reliable (to put it mildly). Knowing what area the model is "looking at" is virtually meaningless if you don't know what it's looking at. We already knew it was looking at the brain so is it turtles all the way down? Those maps are great for confirming what people want to see "oh look it tells a cat from a dog by the eyes" but have no explanatory power. If they did then the paper would be about AI finding a physiological difference and would tell us what it is, not some handwaving about "here's where it looked".
- drdeca 3y ago> Knowing what area the model is "looking at" is virtually meaningless if you don't know what it's looking at. We already knew it was looking at the brain so is it turtles all the way down? Huh? I’m not sure what you are saying here. If there’s some subset of the pixels or whatever which the models predictions depend on more than the others, then, it depends on those more than it depends on the others. Obviously it would be using the data it is given. That, doesn’t mean that looking into “what part of the data that it is given, is it using?” not meaningful.