> Apparently function approximation is an important property 'important' sure but not always a good idea. When you are learning from finite (but potentially ve

> Apparently function approximation is an important property 'important' sure but not always a good idea. When you are learning from finite (but potentially very large) number of noisy examples, the universal function approximator will try to approximate the noise also. Learning is a delicate dance between the capacity of your learner and the complexity of the thing that you are trying to learn. The key notion is that it takes two to tango. If you have few examples, you are better of not using deep nets. Stated another way, a universal approximator has infinite potential for distraction. "Hey shiny" and it could veer off in a direction you don't want it to go. You want to pick out army tanks from pictures and it might learn to distinguish pictures taken on a cloudy day from those taken on sunny days.