Wow. As far as I know, this is the first time anyone reputable[a] has claimed to show (!) that the "manifold hypothesis" is the fundamental principle that makes

Wow. As far as I know, this is the first time anyone reputable[a] has claimed to show (!) that the "manifold hypothesis" is the fundamental principle that makes deep learning work, as has long been believed: "In this work, we give a geometric view to understand deep learning: we show that the fundamental principle attributing to the success is the manifold structure in data, namely natural high dimensional data concentrates close to a low-dimensional manifold, deep learning learns the manifold and the probability distribution on it." Moreover, the authors also claim to have come up with a way of measuring how hard it is for any deep neural net (of fixed size) to learn a parametric representation of a particular lower-dimensional manifold embedded in some higher-dimensional space: "We further introduce the concepts of rectified linear complexity for deep neural network measuring its learning capability, rectified linear complexity of an embedding manifold describing the difficulty to be learned. Then we show for any deep neural network with fixed architecture, there exists a manifold that cannot be learned by the network." Finally, the authors also propose a novel way to control the probability distribution in the latent space. I'm curious to see how their method compares and relates to recent work, e.g., with discrete and continuous normalizing flows: "...we propose to apply optimal mass transportation theory to control the probability distribution in the latent space." This is not going to be a light read... -- [a] One of the authors, Shing-Tung Yau, is a Fields medalist: https://news.ycombinator.com/item?id=18987219 https://news.ycombinator.com/item?id=18987219