This is still a long way away from truly understanding the representational power of deep nets, but it's the first general progress that has been made since the

This is still a long way away from truly understanding the representational power of deep nets, but it's the first general progress that has been made since the early 90's. Exactly. Someone should tell Yann LeCun this --see e.g. Section 3.2 in [1], or pretty much every time he brings up circuit complexity theory to automagically imply that "most functions representable compactly with a deep architecture would require a very large number of components if represented with a shallow one." [ibid, p.14] [1] http://yann.lecun.com/exdb/publis/pdf/bengio-lecun-07.pdf http://yann.lecun.com/exdb/publis/pdf/bengio-lecun-07.pdf