3 ms·

The way I took it is that CNNs use local context and are translation neutral (so if a certain combination of pixels is an eye, they'll be an eye anywhere in the

The way I took it is that CNNs use local context and are translation neutral (so if a certain combination of pixels is an eye, they'll be an eye anywhere in the image), while RNNs can decide which information to keep and can decide in what way positioning affects result. So the thing CNNs are really good at are perfect for small images, but symbols already have that thing done. When CNNs and RNNs are used together (e.g. image segmentation), it's almost as if the CNN is creating symbols and RNN s processing them. Or, you know, maybe I'm far off. I'm in no way qualified to talk about this (just a fan).

sdenton4 9y ago
Here's another thing to consider. The brain is processing "video" in what we might call 3.5 dimensions. (2d images + time with around 6 channels... Plus sound). Image recognition with a CNN is working on still images, and dropping the time component entirely. RNNs are about handling sequential data, discrete or not... Meanwhile, the hardware in our heads is far more complex and capable than the models we're building. And we're not even sure how that hardware works, or what principles we can abstract out and simplify. A neuron getting electrical and hormonal signals is far more complex than an lstm cell... Does it need to be? Or are there biological requirements built in which we don't understand? And how many evolutionary accidents which never evolved away?