The first time I saw a paper on feasible deep networks was at NIPS 2006, specifically this paper: http://www.cs.toronto.edu/~hinton/absps/fastnc.pdf http://www.

The first time I saw a paper on feasible deep networks was at NIPS 2006, specifically this paper: http://www.cs.toronto.edu/~hinton/absps/fastnc.pdf http://www.cs.toronto.edu/~hinton/absps/fastnc.pdf It's been awhile since I read the paper, but as I recall it involved training an unsupervised model layer-by-layer (training a layer, freezing the weights, then training another layer on top of it).