3 ms·
The difficulty here is operationalizing the terms involved to make the point of semantic domain vs pixel domain meaningful at all. But I think this point makes
by hackinthebochs 9y ago
The difficulty here is operationalizing the terms involved to make the point of semantic domain vs pixel domain meaningful at all. But I think this point makes it a substantive distinction: have a representation in the semantic domain with respect to the content of the image.
There are various semantic features of an image and we presume that our visual system is extracting and operating on these semantic features. So it seems important to ask whether the features causing the misclassification can be represented in a semantic domain of the image. Shapes, (relative) sizes, gradients, high level patterns, etc are examples of semantic domains. There seems to be a distinction between the kinds of optical illusions humans are susceptible to and the kinds CNNs are. In the graffiti illusions, the placement of the image that causes our 3D object recognition system to kick in can be described through angles, lines, focal points, focal lengths, etc. These are all semantic features of the scene depicted in an image. Contrast this with CNN adversarial perturbations which have zero semantic features that we recognize. This seems like an important result. It means that our visual system is robust against certain classes of adversarial images, namely the small imperceptible deltas that trip up CNNs. To trip up our visual system requires certain combinations of semantic input which are harder to exploit (larger delta is harder to exploit).
- taeric 9y agoThen I offer the ease with which we see faces where they aren't. The very constellations? I get your point to an extent. However, I think it is presented stronger than it is. Specifically, the semantics of imagery are not much more than non frequency analysis of the pixel data your eye sees.