4 ms·

What's different between an "embedding" and a projection, which I believe is the more standard term for this kind of transformation?

• thanatropism 8y ago
  Projection is a type of embedding. But you can't really describe what LTSA, UMAP, etc. do as projection. LTSA "unrolls" data rather than projecting it.
  • xapata 8y ago
    On the contrary, I'd say an "unrolling" is a non-linear projection. Wikipedia suggests that embedding is a type of projection. I hadn't heard the term before today. https://en.wikipedia.org/wiki/Nonlinear_dimensionality_reduction#Modified_Locally-Linear_Embedding_(MLLE) https://en.wikipedia.org/wiki/Nonlinear_dimensionality_reduc...
• mlucy 8y ago
  "Embedding" is the term I've heard used for this most often. It's definitely the term that seems to dominate in the literature. (Just to pick a random paper off my reading list: https://arxiv.org/abs/1709.03856 https://arxiv.org/abs/1709.03856 .) In my mind "embedding" carries the connotation that you're moving into a smaller space that's easier to work with, and where things which are similar in some way are near each other.
  • xapata 8y ago
    The TensorFlow docs agree with you. This field has such jargon proliferation, it's hard to keep up. https://developers.google.com/machine-learning/crash-course/embeddings/video-lecture https://developers.google.com/machine-learning/crash-course/...
    • mlucy 8y ago
      Yeah, I agree. The number of new nouns per year is kind of ridiculous.
• sjg007 8y ago
  Embedding is the ML term for a non-linear projection.
  • xapata 8y ago
    Amusingly, it seems you and the other reply are contradicting each other, saying both that an embedding is a specific type of projection and that a projection is a specific type of embedding. When I was learning about SVMs way back when, we said "non-linear projection" instead of "embedding".
    • sjg007 8y ago
      I mean both are correct given the literature at this point. Maybe the use of embedding captures the notion that you are mapping through a NN which is a variable function of sorts instead of a single or set of linear or orthonormal functions.
      • FridgeSeal 8y ago
        The mathematical term "Embedding" refers to finding a lower dimensional representation of some high(er) dimensional data, whether this is done with single function (maybe hand chosen) or a neural network doesn't matter.
        • sjg007 8y ago
          I don't think that's quite right.. According to wikipedia, embedding is: https://en.wikipedia.org/wiki/Embedding https://en.wikipedia.org/wiki/Embedding So Projection sounds closer to what you describe. https://en.wikipedia.org/wiki/Projection_(mathematics) https://en.wikipedia.org/wiki/Projection_(mathematics) So my original comment is not quite right either. The embedding is a structure preserving map. A projection isn't necessarily structure preserving. ''' To be an embedding, such a mapping must preserve order "both ways": ''' http://mathworld.wolfram.com/Embedding.html http://mathworld.wolfram.com/Embedding.html I think that a projection from p dimensions to q<p dimensions isn't structure preserving. At this point a mathematician may be able to explain more.