3 ms·

To play devil's advocate... given that manifolds are so general, wouldn't it be more surprising if natural data didn't form some lower dimensional manifold? Th

To play devil's advocate... given that manifolds are so general, wouldn't it be more surprising if natural data didn't form some lower dimensional manifold? That is, a manifold only requires some consistent notion of neighborhoods and a way to locally parameterize objects, which is a fairly low bar to pass, and perhaps trivial if you knock off one pixel in the case of images. Maybe the surprising point (which is not said explicitly) is that the manifold is very low dimensional compared to sensor data, but I suppose it's hard to formulate a hypothesis about the magnitude of the reduction besides it being "lower".

throwawaymath 8y ago
> To play devil's advocate... given that manifolds are so general, wouldn't it be more surprising if natural data didn't form some lower dimensional manifold? No, because not all data can actually be represented on a topology. It's nice if you can model your data as a normed vector space, but it's not inherently possible in general.
- cafebeen 8y ago
  In a trivial sense, it seems that a Euclidean topology would do the trick, no? I'd be curious to hear counter-examples, of course. In my mind, I suppose what's missing from the "manifold hypothesis", as stated above, is that the manifolds should be more useful than raw data, for example, does their structure respect our notion of object categories or are they sufficiently low-dimensional for visualization?
  - srean 8y ago
    They key is the low dimensional part not that it is a manifold. The ambient space may be huge, but data is not spread all over but lies on a tiny subset that can be well described by very few parameters. This limited degree of freedom in the data is what makes it easy to learn. A priori there is no reason why data should have such a property. I would be interested in your thoughts on how you would map human language understanding to a Euclidean topology.
jvkersch 8y ago
A counterexample would be a dataset that forms a fractal in the ambient space. I don't know of a realistic example of this, but it seems plausible if you think of scale-invariant phenomena. Other ways of getting fractal-like structures, or at least "dust-like" structures with weird topologies, is by taking intersections of smooth structures. These things would be hard to separate...

throwawaymath 8y ago

> To play devil's advocate... given that manifolds are so general, wouldn't it be more surprising if natural data didn't form some lower dimensional manifold? No, because not all data can actually be represented on a topology. It's nice if you can model your data as a normed vector space, but it's not inherently possible in general.

cafebeen 8y ago
In a trivial sense, it seems that a Euclidean topology would do the trick, no? I'd be curious to hear counter-examples, of course. In my mind, I suppose what's missing from the "manifold hypothesis", as stated above, is that the manifolds should be more useful than raw data, for example, does their structure respect our notion of object categories or are they sufficiently low-dimensional for visualization?
- srean 8y ago
  They key is the low dimensional part not that it is a manifold. The ambient space may be huge, but data is not spread all over but lies on a tiny subset that can be well described by very few parameters. This limited degree of freedom in the data is what makes it easy to learn. A priori there is no reason why data should have such a property. I would be interested in your thoughts on how you would map human language understanding to a Euclidean topology.

jvkersch 8y ago

A counterexample would be a dataset that forms a fractal in the ambient space. I don't know of a realistic example of this, but it seems plausible if you think of scale-invariant phenomena. Other ways of getting fractal-like structures, or at least "dust-like" structures with weird topologies, is by taking intersections of smooth structures. These things would be hard to separate...