4 ms·

Thanks for the reminder. I was thinking of this 1991 paper, which I ran into a long time ago: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.41.192 h

by mturmon 14y ago

Thanks for the reminder. I was thinking of this 1991 paper, which I ran into a long time ago: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.41.192 http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.41.1... They used a (linear) "neural network" with gradient descent training that implemented PCA (kind of an iterative graham-schmidt process), and got Gabor-like filters. I think a lot of people have done similar experiments, with varying results.

psb217 14y ago
I hadn't seen that paper before; thanks for the reference. I read through it and saw that they were reweighting the sampled image patches with a Gaussian mask prior to learning, which explains how they got Gabor-like filters. The masking effectively forced the learned filters to have localized receptive fields, while locality/nonlocality is generally one of the (visually) clearer differences between filters learned with ICA/PCA. In other words, the Gaussian-modulated part of Gaussian-modulated sinusoids was built into their learning process, rather than appearing as an emergent property. I also chuckled a bit when they described how computing eigenvectors for 4096x4096 matrices was "beyond reasonable computation".