5 ms·
The universal approximator approach is another big one that people lean on from the opposite direction (I suppose you could call it the "anti-deep learning" sch
by clmcleod 10y ago
The universal approximator approach is another big one that people lean on from the opposite direction (I suppose you could call it the "anti-deep learning" school of thought). Most people don't take the time to consider what a universal approximator actually is.
Are humans universal approximators? Absolutely not. The mere proof that a nn with an infinite number of hidden nodes (which isn't practical, which could end the discussion there) can find a statistical mapping between any two input and outputs is hardly a benefit. There is something so much more to intelligence --- the ability to generalize well (aka not overfitting) is first and foremost.
I'd push you to ask why you perceive that stacking RBMs produces "eerily human-like" results. I'd argue that they don't: http://blog.keras.io/how-convolutional-neural-networks-see-the-world.html http://blog.keras.io/how-convolutional-neural-networks-see-t...
- return0 10y agoTrying to compare anything to "intelligence" is a trap, imho. Intelligence is not a natural quantity, it's a word. There was no reason for a brain to become a universal approximator, or to have any specific mathematical property for that matter. Brains evolved to adapt behavior to the environment using the senses. The universe didn't care for creating some property called "intelligence". I do see many results in NLP for example that have an eerie human-likeness, even if nobody can explain why. I am not an expert in the field, but i would think it's a challenge to find similar results made using another technique.
- cubano 10y ago> The universe didn't care for creating some property called "intelligence". On the face of it, I would have to disagree. It would seem to me that's exactly what the universe selected for, among many other things of course, in our tiny little corner of it. Otherwise, we wouldn't be here debating it.
- return0 10y agoDebating the anthropic principle will be left for another thread.
- kmiroslav 10y agoMaybe this universe did select for intelligence. How about all the other universes that never generated intelligence? But going back to our own universe, how widespread is intelligence? It's pretty much microscopic, isn't it? I think our universe seems to be mostly geared toward generating empty space, and the rare areas that are not empty are certainly extremely hostile to intelligence.
- visarga 10y agoPredicting (function approximation) is perhaps the main use of neural nets but there is another very important use: generating, or in other words, imagination. And in generative mode we need to know about probabilities and latent variables - thus, we need a little more than neural nets to do this task.