3 ms·
There's some truth to this, but it's not the whole story. CNNs have limitations in terms of what they can represent and what they can learn. Counting is an ex
by cameldrv 7y ago
There's some truth to this, but it's not the whole story. CNNs have limitations in terms of what they can represent and what they can learn. Counting is an example of this. If I create a class of monsters with 10 eyes, and another that is otherwise identical, but with 11 eyes, these will be difficult classes to tell apart.
When it comes to "shape", part of the problem is definitional. In the MNIST example, the network will fairly easily learn features that are nodes, like where two lines intersect, but it has trouble with paths, like "an unbroken line that smoothly intersects itself." CNNs have trouble distinguishing things like a spiral from a set of concentric circles because locally they look the same.
- gjm11 7y ago> CNNs have trouble distinguishing things like a spiral from a set of concentric circles because locally they look the same. So does the human visual system; see e.g. https://en.wikipedia.org/wiki/Fraser_spiral_illusion https://en.wikipedia.org/wiki/Fraser_spiral_illusion .
- petters 7y agoIt seems to me that the number of eyes problem should be solvable. Just convolve with an eye detector and threshold appropriately. Then feed everything to a single neuron in the next layer and threshold at 10.5. No? The spiral vs. circles I agree with.
- mkl 7y agoSolvable with human guidance, yes, but would any existing machine learning system come up with a successful strategy like this by itself? Most deep dream style images suggest the networks don't even realise that dogs have two eyes.