3 ms·

Amusingly, it seems you and the other reply are contradicting each other, saying both that an embedding is a specific type of projection and that a projection i

by xapata 8y ago

Amusingly, it seems you and the other reply are contradicting each other, saying both that an embedding is a specific type of projection and that a projection is a specific type of embedding. When I was learning about SVMs way back when, we said "non-linear projection" instead of "embedding".

sjg007 8y ago
I mean both are correct given the literature at this point. Maybe the use of embedding captures the notion that you are mapping through a NN which is a variable function of sorts instead of a single or set of linear or orthonormal functions.
- FridgeSeal 8y ago
  The mathematical term "Embedding" refers to finding a lower dimensional representation of some high(er) dimensional data, whether this is done with single function (maybe hand chosen) or a neural network doesn't matter.
  - sjg007 8y ago
    I don't think that's quite right.. According to wikipedia, embedding is: https://en.wikipedia.org/wiki/Embedding https://en.wikipedia.org/wiki/Embedding So Projection sounds closer to what you describe. https://en.wikipedia.org/wiki/Projection_(mathematics) https://en.wikipedia.org/wiki/Projection_(mathematics) So my original comment is not quite right either. The embedding is a structure preserving map. A projection isn't necessarily structure preserving. ''' To be an embedding, such a mapping must preserve order "both ways": ''' http://mathworld.wolfram.com/Embedding.html http://mathworld.wolfram.com/Embedding.html I think that a projection from p dimensions to q<p dimensions isn't structure preserving. At this point a mathematician may be able to explain more.
    - xapata 8y ago
      SVMs project the data into a higher dimensional space, not lower. A mixed blob of dots on a plane can't be cleanly separated by a line into two colors, but if you blow them out into a cube by some arbitrary projection, a plane will be able to separate them cleanly.

by xapata 8y ago

sjg007 8y ago

I mean both are correct given the literature at this point. Maybe the use of embedding captures the notion that you are mapping through a NN which is a variable function of sorts instead of a single or set of linear or orthonormal functions.

FridgeSeal 8y ago
The mathematical term "Embedding" refers to finding a lower dimensional representation of some high(er) dimensional data, whether this is done with single function (maybe hand chosen) or a neural network doesn't matter.
- sjg007 8y ago
  I don't think that's quite right.. According to wikipedia, embedding is: https://en.wikipedia.org/wiki/Embedding https://en.wikipedia.org/wiki/Embedding So Projection sounds closer to what you describe. https://en.wikipedia.org/wiki/Projection_(mathematics) https://en.wikipedia.org/wiki/Projection_(mathematics) So my original comment is not quite right either. The embedding is a structure preserving map. A projection isn't necessarily structure preserving. ''' To be an embedding, such a mapping must preserve order "both ways": ''' http://mathworld.wolfram.com/Embedding.html http://mathworld.wolfram.com/Embedding.html I think that a projection from p dimensions to q<p dimensions isn't structure preserving. At this point a mathematician may be able to explain more.
  - xapata 8y ago
    SVMs project the data into a higher dimensional space, not lower. A mixed blob of dots on a plane can't be cleanly separated by a line into two colors, but if you blow them out into a cube by some arbitrary projection, a plane will be able to separate them cleanly.